Page 27 of 129
Open with How many users are there? and What is the expected read to write ratio? For a detail you are parking: I'm going to put a pin in that. For a cost: …but it will come at the cost of latency. After comparing, commit: I'm going to go with an event queue because I'd rather… After a section, ask Do you think that makes sense?
Design controlled comparisons, retain paired outcomes, and distinguish replication from working code in a generative-model project.
Knowing that backpropagation uses the chain rule is different from implementing a correct training loop.
Separate paper claims, empirical support, and your own interpretation through a role-based report and a controlled-comparison example.
An RNN compresses the prefix into a state.
Learning a next-token distribution and choosing a token from that distribution are consecutive but distinct operations.
Lecture 7 constructed noisy inputs and noise targets.
A conventional autoencoder compresses an input into a vector and reconstructs it.
Separate what ICL, SFT, reward models, and RLHF each change, with worked examples and answered variations.
Derive the DPO loss with a recomputable example, then trace shapes and conditioning paths in latent diffusion.
A complete worked example and answered variations for Vision-Language Models: Turning Images into Language-Model Inputs.
A complete worked example and answered variations for Cross-Attention and DiT: How Conditions Enter a Denoiser.
Trace Q-Former queries and two-stage training, then compute a model/data split under a fixed compute budget.
A complete worked example and answered variations for Mixture of Experts: Separating Capacity from Active Computation.
A complete worked example and answered variations for Distributed Training: Splitting Data, Models, and Optimizer State.
A complete worked example and answered variations for Efficient Inference: FlashAttention, KV Caches, and Speculative Decoding.
Long Context: Fitting Input Is Not Reliable Retrieval | CMU 10-423 Lecture 19
State Space Models: From Recurrence to Convolution | CMU 10-423 Lecture 21
Generative AI Risks: Turning Failures into Tests | CMU 10-423 Lecture 22
Compute frames, mel bins, and WER, then account for codec tokens and contrast diffusion sampling.