Testing LoRA Fine-Tuning: Gradients, Data, and Evaluation | CMU 10-423 Assignment 3
Use an original numerical test to check gradients, data, and evaluation alongside the original assignment specification.
#peft
7 postsPEFT and ICL: Change Parameters or Change the Input | CMU 10-423 Topic Guide
Adapting without retraining every weight can mean two different things. LoRA-style PEFT still trains some parameters. ICL supplies examples in context without updating parameters during that inference. The source spans the end of Lecture 9, Lecture 10, and the ICL material in Lecture 11.
Efficient Fine-Tuning for Large Models: LoRA, CIAT, and QLoRA | CMU 11-868 Lecture 23
Lecture 23 of 11-868 treats fine-tuning as a memory problem. Full-parameter half-precision fine-tuning of LLaMA-8B needs about 80GB. LoRA brings that to about 33GB, and QLoRA, which stores the frozen weights in 4 bits, gets it to about 9.2GB. The lecture moves in three steps: train only two small low-rank matrices A and B (the slides credit CIAT as the first to do this); squeeze frozen weights to about 0.52 bytes per parameter with an NF4 lookup table and double quantization; and use a paged optimizer to push optimizer state to the CPU when the GPU is about to run out.
LLM Post-Training: From SFT and RLHF to Fine-Tuning That Touches 1% of the Weights | MIT 6.5940 Lecture 14
Lecture 14 has three parts. Fine-tuning: SFT runs next-token prediction on desired answers, RLHF trains a reward model and then fine-tunes with KL-penalized RL, and DPO collapses both stages into one supervised step. Then comes a chain of PEFT methods: BitFit tunes only biases, Adapters add small layers but slow inference, Prompt/Prefix-Tuning eat input length, LoRA fixes latency with a low-rank branch you can merge back, QLoRA stores the backbone in NF4, and BitDelta compresses the fine-tune delta to 1 bit. Multimodal LLMs: Flamingo uses cross-attention, PaLM-E and VILA feed images in as tokens, and VILA-U can also output images. Prompt engineering: zero/few-shot, CoT, and RAG.
Reading NTHU Kao's NLP: Parameter-Efficient Fine-Tuning — Fine-Tuning Large Models Without an A100 Cluster | NTHU Hung-Yu Kao Natural Language Processing Topic Guide
Hung-Yu Kao's Fall 2025 PEFT slides open with a budget: full fine-tuning of Llama 2-7B in 16-bit needs about 56GB of GPU memory, while training only 0.2M parameters brings it down to about 17GB, because gradients and optimizer states nearly vanish. Intrinsic dimensionality then explains why tuning a small slice is enough: the longer a model is pretrained and the larger it is, the fewer effective dimensions fine-tuning needs. Methods fall into additive (Adapters, Prompt Tuning), selective (BitFit), reparametrization (LoRA), and hybrid (MAM Adapters, S4). The second half runs from GPT-2's task descriptions and verbalizers to the trade-offs between prefix tuning and soft prompt tuning.
Post-Training and Efficient Adaptation: How a Model Learns to Follow Instructions and How to Tune It Cheaply | NTU Yun-Nung Chen Applied Deep Learning Lecture 7
A pre-trained model continues text well and still may not do what an instruction asks. The first half of this lecture covers the two steps of post-training: instruction tuning uses many different tasks to teach the model to read task descriptions, and RLHF then moves the model toward whole responses that people prefer, with a reward model and pairwise comparisons handling the cost of human labor and inconsistent scores, while DPO and KTO drop the reinforcement-learning part. InstructGPT and ChatGPT are these two steps chained together. The second half covers LLM adaptation: models are too large to fine-tune in full, so only a small number of parameters are trained, with Adapter, LoRA, and prompt tuning, and no single method is best on every task.
Prompting, LoRA, and Parameter-Efficient Adaptation | Stanford CS224N Lecture 9
Lecture 9 compares prompting, pruning, LoRA, prompt tuning, and adapters. Each asks the same question: how many parameters must change, and how much task-specific state must be stored, to adapt a large pretrained model?