Final Projects: Build a Reproducible Generative-Model Experiment | CMU 10-423 Topic Guide
Design controlled comparisons, retain paired outcomes, and distinguish replication from working code in a generative-model project.
Design controlled comparisons, retain paired outcomes, and distinguish replication from working code in a generative-model project.
Knowing that backpropagation uses the chain rule is different from implementing a correct training loop.
Separate paper claims, empirical support, and your own interpretation through a role-based report and a controlled-comparison example.
An RNN compresses the prefix into a state.
Learning a next-token distribution and choosing a token from that distribution are consecutive but distinct operations.
Lecture 7 constructed noisy inputs and noise targets.
A conventional autoencoder compresses an input into a vector and reconstructs it.
Separate what ICL, SFT, reward models, and RLHF each change, with worked examples and answered variations.
Derive the DPO loss with a recomputable example, then trace shapes and conditioning paths in latent diffusion.
A complete worked example and answered variations for Vision-Language Models: Turning Images into Language-Model Inputs.
A complete worked example and answered variations for Cross-Attention and DiT: How Conditions Enter a Denoiser.
Trace Q-Former queries and two-stage training, then compute a model/data split under a fixed compute budget.
A complete worked example and answered variations for Mixture of Experts: Separating Capacity from Active Computation.
A complete worked example and answered variations for Distributed Training: Splitting Data, Models, and Optimizer State.
A complete worked example and answered variations for Efficient Inference: FlashAttention, KV Caches, and Speculative Decoding.
Long Context: Fitting Input Is Not Reliable Retrieval | CMU 10-423 Lecture 19
State Space Models: From Recurrence to Convolution | CMU 10-423 Lecture 21
Generative AI Risks: Turning Failures into Tests | CMU 10-423 Lecture 22
Compute frames, mel bins, and WER, then account for codec tokens and contrast diffusion sampling.
Video Generation: Modeling the Time Axis | CMU 10-423 Lecture 25
Separate three world representations and evidence of action control, then examine consensus costs and corrigibility theorem conditions.
A complete worked example and answered variations for LoRA: What Changes When Most Weights Stay Frozen.
Use the public practice exam to diagnose assumptions, data flow, and evaluation errors through a worked probability example.
AI-901 replaces AI-900, retired June 30, 2026. The certification is still called Azure AI Fundamentals, but the objectives are now two areas: concepts at 40–45% and implementing with Microsoft Foundry at 55–60%. Python is an explicit requirement, and the objectives include building a lightweight chat client with the Foundry SDK and creating a single agent. Official specs: $99 in the US, $50 in Taiwan, 45 minutes, pass at 700, 13 languages including Traditional Chinese, and fundamentals certifications never expire.
Code Generation and Agents: A Reliable Test–Repair Loop | CMU 10-423 Lecture 23
A GAN learns from a discriminator.
A classifier asks which class an image belongs to.
This series uses the public Spring 2026 materials for CMU 10-423/623/723 Generative AI.
Copying a noising equation is not enough to implement diffusion correctly.
Reasoning Models: Spending Compute on Verifiable Answers | CMU 10-423 Lecture 20
Diffusion and VAEs share probabilistic ideas without sharing the same generation procedure. Diffusion chains noise steps; a VAE approximates a latent posterior with an encoder and reconstructs through a decoder. This guide connects those ideas while directing each mechanism to its own lesson.
MIT 6.S184 is a short IAP (January Independent Activities Period) course: five lectures (Lecture 3 is split into two recordings, 3-A and 3-B), three labs, and an 84-page set of lecture notes the course calls its backbone. Notes, slides, all six recordings, lab notebooks, and official solutions are public, so it grades A3, enough to self-study. Two gaps remain: lab submission goes through Gradescope inside Canvas, which only enrolled MIT students can use, and Lecture 5 on discrete diffusion has no lab.
MIT 6.S184 Lab 1 has three parts. First you write the step functions for Euler and Euler–Maruyama. Then you use them to simulate Brownian motion and the Ornstein–Uhlenbeck process and watch how σ and θ shape trajectories and the final distribution. Finally you implement Langevin dynamics, watch a cloud of points get pushed toward a five-mode Gaussian mixture, and show by hand that the OU process is Langevin dynamics with a Gaussian target. The questions, code scaffolding, and official solutions are all on GitHub; outside readers get no Gradescope grading and have to check against the solutions themselves.
Lab 3 builds a conditional latent diffusion model on MNIST from scratch, in four stages: CFG training with label dropout (checked on a three-component Gaussian mixture), a diffusion transformer built piece by piece (Fourier time embedding, patchify, multi-head attention, adaLN-Zero, depatchify), a VAE, and finally the DiT trained inside the VAE's latent space. Problems and official solutions are public; submission goes through Gradescope on Canvas, which only enrolled MIT students can use.
Lecture 1 of MIT 6.S184 first rewrites "generate an image of a dog" as "sample from the data distribution," then gives the machine that does the sampling: start from Gaussian noise and simulate an ODE along a neural-network vector field (a flow model), or add a little Brownian-motion noise at every step to get an SDE (a diffusion model). Each is simulated with the simplest numerical method available, Euler and Euler–Maruyama. How to train the vector field is left to Lecture 2.
The object we want is the marginal vector field: run an ODE along it and noise flows into data. The catch is that it requires an integral over the whole dataset, so we can't compute it. Flow matching regresses on the conditional vector field instead, the one that pushes noise toward a single data point, which has a closed form. Theorem 12 in the notes shows the two losses differ by a constant and share the same gradient. On the CondOT path, training reduces to one line: sample data z, noise ε, and time t, and have the network predict z − ε at the point tz + (1−t)ε.
A score function is the gradient of the log density; it points toward where probability rises fastest. On Gaussian paths, the score and last lecture's vector field are both linear in x and z, so each converts into the other (Proposition 1 in the notes): learn one and you have learned both. With the score in hand, you can add noise of any strength to the ODE and turn it into an SDE without changing the distribution at any time (Theorem 17). The score itself is learned with the same trick as flow matching, by regressing on the conditional score. On Gaussian paths, that amounts to predicting the noise that was added, which is the DDPM training objective.
Feeding the prompt to the network as an extra input should, in theory, sample from p_data(x|y), but in practice the images don't follow the prompt closely enough. Lecture 3B uses Bayes' rule to split the guided vector field into the unguided vector field plus a classifier gradient; scaling that classifier term by w is classifier guidance. Replacing the classifier with the difference between guided and unguided fields gives CFG, which needs no classifier: ũ = (1−w)·u(x|∅) + w·u(x|y). Training only requires swapping the label for a null label ∅ with probability η. The costs: two network calls per step, and for w>1 you are no longer sampling from the data distribution.
The algorithms are complete by Lecture 3B; Lecture 4 tackles two engineering problems that show up at scale. First, the network must take an image, a time t, and a prompt and output a vector field of the same size, so we use a U-Net or a diffusion transformer (DiT), embedding time with Fourier features and text with frozen CLIP/T5 encoders. Second, pixel space is too big, so we first train a VAE to compress images into a latent space, run flow matching there, and decode at the end. Stable Diffusion 3 and Meta Movie Gen Video both follow this recipe: flow matching in latent space, a DiT variant, and CFG.
Text is a sequence of discrete tokens. There is no direction to move in, so ODEs and SDEs do not exist. Lecture 5 carries the recipe from Lectures 1–4 over unchanged and swaps only the underlying stochastic process: vector fields become rate matrices, ODEs become continuous-time Markov chains (CTMCs), and the continuity equation becomes the Kolmogorov forward equation. With the factorized mixture path, the marginal rate matrix has exactly one unknown: the probability of each position's original token given the noisy sequence. Training a discrete diffusion model therefore reduces to per-position classification with a cross-entropy loss. Make the noise all [mask] tokens and you get a masked diffusion language model.
The first half of lecture 1 covers course rules and lightning talks. The middle answers "why learn the principles?": Yen-Lung Tsai splits the anxiety of learning AI into three kinds and argues that knowing the principles tells you a model's limits, so you stop chasing every new tool. The second half is a Colab primer, from magic commands and the four standard import lines to plt.plot, Markdown, and ipywidgets. Homework 1 is to plot a function in Colab. On the Chang Gung satellite rubric, a tweaked copy of the demo earns 6 points; a function not taught in class, with well-written Markdown notes, earns 10.
The prompt "a cute girl" has countless correct pictures, so training it as a function only teaches the model the average of all of them. GANs sidestep this by training two networks: a generator G turns a random latent vector into an image, a discriminator D judges real versus fake, and the two compete. L03 walks from the 2014 paper through WGAN, Progressive GAN, StyleGAN's 512-dimensional latent and AdaIN, then Pix2Pix and CycleGAN. An appendix explains cross entropy and KL divergence as a 'surprise index'. Week 3 homework: run a GAN yourself, or explain CE and KL in your own words.
L04 reduces a large language model to one sentence: look at the preceding words, score every word in the vocabulary, turn the scores into probabilities with softmax, and sample the next word. To give the model a memory of what came before, the lecture covers RNNs and then gives a first look at Transformer Q/K/V. GPT-2's 1.5 billion and GPT-3's 175 billion parameters illustrate scale; temperature and top-p explain why every answer comes out different. The second half covers running open models locally and estimating VRAM. Week 4 homework: write test prompts on a topic you know well and compare at least two LLMs.
L05 reads the whole Transformer with two linear-algebra rules: matrix multiplication is row-times-column dot products, and a row vector times a matrix is a linear combination of the matrix's rows. With those, attention is 'dot the query with every key, softmax into weights, take a weighted average of the values,' or softmax(QKᵀ/√d_k)V in batch form. Dividing by √d_k just pulls the numbers back toward 0 so softmax doesn't become winner-take-all. Then come multi-head attention, encoder versus decoder, masking, positional encoding as a set of sin/cos clocks, and ResNet-style residuals with layer normalization. No homework this week.
The first half of L06 is about ethics. Yen-Lung Tsai quotes Karpathy's line that hallucination is a feature of LLMs, then works through plagiarism, whether your data gets used for training, and DeepSeek's censorship and corpus skew, and closes with seven principles of responsible use. The second half is about applications: give the model the right information and clear instructions, and one system prompt becomes a Lucky Vicky positivity generator, a social-media copywriter, or a biased college-major counselor. The week-6 assignment moves that prompt into an OpenAI-compatible API with a Gradio front end: a chatbot with a persona.
A chatbot 'remembers' you not because the model has memory, but because your code resends the whole messages list (system, then alternating user and assistant) every turn. L07 starts with getting OpenAI and Groq keys, spells out that structure, then runs Gemma 3 locally or in Colab with Ollama, where the same openai package works after changing only base_url. The week-7 assignment offers two options: a version that keeps the conversation going, or two models talking to each other, both demoed in Gradio.
L06 said a prompt is two things: correct information and clear instructions. RAG lets the computer fetch the information part on its own. Split your documents into chunks, turn chunks and questions into feature vectors with the same model fθ, find the closest few chunks, and drop them into a template: 'Answer {question} based on {retrieved_chunks}.' The code comes in two notebooks: Demo06a builds a vector database with LangChain and FAISS and zips it as faiss_db.zip; Demo06b loads it back, connects an LLM, and wraps it in Gradio. The week-8 assignment is to do the same with your own data.
L09 defines an AI agent in one line: the AI finishes the work you would otherwise do yourself. Yen-Lung Tsai follows Andrew Ng's four design patterns (Reflection, Tool Use, Planning, Multiagent Collaboration) but builds only the two easiest. Demo07a hands a draft between a "writer" and a "reviewer" LLM call. Demo07c splits the Lucky Vicky post generator into "think of five reasons, then write the post", a two-stage CoT. Both use AISuite with Groq and a Gradio front end. LangChain, AutoGen and CrewAI appear only on a further-learning list. The week 9 homework asks you to pick one of the two patterns.
L10 starts from one question: how do you find a good feature vector? Word2Vec learns embeddings through a pretext task. An autoencoder squeezes out a latent vector by being forced to reproduce its input. A VAE then asks the latent to follow a normal distribution, so nearby points produce similar images. Yen-Lung Tsai then recasts diffusion as "an autoencoder whose encoder is computed and whose decoder is learned", and ends on latent diffusion: a VAE shrinks a 512×512 image to 64×64, and diffusion runs only in that small space. The week 10 homework involves no code: make several style-consistent image sets with Bing.
L11 fills in the rest of the Stable Diffusion diagram. CLIP is trained so that matching text and images get similar vectors, which turns a prompt into a 77×768 embedding. Schedulers compress 1,000 noising steps into twenty or thirty denoising steps, but ancestral samplers such as Euler a never settle: push to 100 steps and the subject changes jackets and seats. LoRA freezes the original W and learns only a ΔW factored into A·B. The hands-on part loads an SD 1.5-family model with diffusers, and the week 11 homework is your own image-generation web app.
The Stable Diffusion setup from L11 listens only to the prompt, so composition and pose are left to luck. L12 adds a steering wheel. ControlNet copies a block of SD and wires the copy back in through zero convolutions, so extra conditions such as edge maps, poses, and depth maps can steer generation. The standard example is Canny edges. The second half covers Fooocus, an SD interface that aims to be 'as simple as Midjourney': Presets, Styles, and the five Input Image features, where Image Prompt is ControlNet with a friendly wrapper. Week 12 homework: pick a use case, make at least 3 image sets in Fooocus, and write up your creative process.
Every model in the first 12 lectures learned from training data that people prepared. L13 asks a different question: when there is no right answer, only a signal of how well you did, how does a computer learn? Tsai starts from AlphaGo and Breakout and splits the field in two. Value-based methods learn a Q function that scores each action (Deep Q-Learning, TD, experience replay, ε-greedy); policy-based methods learn the action directly (policy gradient, actor-critic). The second half returns to LLMs: ChatGPT trains a reward model from human rankings and then runs RLHF with PPO, while DeepSeek has the computer check math answers automatically and uses that as the reward. Week 13 homework is the final project proposal.
The last lecture looks at two lines of technology crossing into each other. LLMs such as ChatGPT have started drawing, and the slides use early fusion plus VQ-VAE/VQGAN to explain how an image can be cut into tokens. Going the other way, Inception Labs' Mercury generates text with diffusion, noising a sentence into a row of [MASK] tokens and then restoring it. Next come a few papers anyone can use: evaluating RAG automatically, reasoning models being easier to hijack, and DeepMind's four kinds of AI risk. The lecture ends with vibe coding and a list of application tools, and the final project runs as an online conference in Gather Town.
Generative AI: Text and Image Synthesis Principles and Practice is an introductory course taught by Yen-Lung Tsai (蔡炎龍) of NCCU's Department of Mathematical Sciences and opened to other schools as a TAICA satellite course. The most complete course page online actually belongs to the Chang Gung University satellite section, where Chih-Yuan Yang is the co-teacher. This series follows Spring 2025 (semester 1132): 14 recordings, 14 slide decks, and 12 homework specs with rubrics are public, and the demo notebooks are on GitHub, so the access grade is A3. The gaps: the notebooks keep changing, submission and grading run through each school's LMS, and final projects were never published.
AIF-C01, MLA-C02, and AIP-C01 are not a difficulty ladder but three job-function slices: AIF tests AI business judgment, MLA tests whether you can put traditional ML, foundation models, and agentic workflows into production, and AIP tests whether you can integrate foundation models into a GenAI system. The MLA-C02 English beta is open for registration; general-availability dates and non-English versions remain unannounced.
The AIF-C01 exam guide moved to v1.1 on April 30, 2026, adding seven objectives — MCP, multi-agent patterns, context engineering, token-based pricing, and hallucination detection all became testable, with Bedrock AgentCore, Kiro, and Strands Agents joining in-scope services. Almost every summary online describes the older version. This guide builds on the official five-domain weighting, integrating AWS's four-step prep method and field-tested advice from 10+ candidates who passed. $100, 90 minutes, 65 questions (50 scored), pass at 700, valid 3 years — and it is offered in Traditional Chinese.
AIP-C01 tests integrating foundation models into production AWS applications: RAG, agents, security, cost, and evaluation. This guide follows the five official domains with a prerequisite self-check, four-step preparation plan, resource choices, and hands-on checkpoints, plus a ten-week schedule, an AI study prompt, and exam-day guidance. Start with a diagnostic, then close gaps through one RAG-and-agent project.
AI-103 replaces AI-102, retired June 30, 2026, and the objectives were rewritten around Microsoft Foundry — prompt flow, Azure AI Studio, Azure OpenAI Service, and Azure AI Agent Service appear nowhere in them. The five skill areas weigh 25-30 / 30-35 / 10-15 / 10-15 / 10-15, with generative AI and agents the largest. Official specs: $165, 120 minutes, pass at 700, offered in Traditional Chinese, may include interactive components — and it is valid for only one year, though renewal is a free, open-book, unproctored online assessment.
NCA-GENL is usually what a job posting means by 'NVIDIA Generative AI / LLM certification.' But the official blueprint diverges sharply from the name — Core Machine Learning and AI Knowledge 30%, Software Development 24%, Experimentation 22%, Data Analysis 14%, Trustworthy AI 10% — with LLM and RAG content scattered at bullet level rather than forming a domain, alongside spaCy, NumPy, Keras, and cross validation. The other thing to know first: the official recommended training was replaced in autumn 2026, and each area now lists just two items, free open-source deep learning course materials on GitHub and one LLM application development course. Official specs: $125, 1 hour, 50–60 items, two-year validity, English only, pass/fail with no score reported.
NCA-GENM matches NCA-GENL on price, length, and level but not on emphasis: Experimentation rises to 25% (the heaviest), Core ML drops from 30% to 20%, and two new areas appear — Multimodal Data 15% and Performance Optimization 10%. The content covers U-Net, CLIP, diffusion models, multimodal loss functions, attention maps, and NVIDIA's Riva / NeMo / Triton / ACE SDKs. The official recommended training was replaced in autumn 2026: it is now one free open-source set of deep learning materials on GitHub plus one multimodal agent course, assigned differently per area, with the heaviest area, Experimentation, getting only the free materials. Official specs: $125, 1 hour, 50–60 items, two-year validity, English only.