#qlora — quidproquo

ai guide Reading CMU 11-868 LLM SystemsSep 30, 2026

Efficient Fine-Tuning for Large Models: LoRA, CIAT, and QLoRA | CMU 11-868 Lecture 23

Lecture 23 of 11-868 treats fine-tuning as a memory problem. Full-parameter half-precision fine-tuning of LLaMA-8B needs about 80GB. LoRA brings that to about 33GB, and QLoRA, which stores the frozen weights in 4 bits, gets it to about 9.2GB. The lecture moves in three steps: train only two small low-rank matrices A and B (the slides credit CIAT as the first to do this); squeeze frozen weights to about 0.52 bytes per parameter with an NF4 lookup table and double quantization; and use a paged optimizer to push optimizer state to the CPU when the GPU is about to run out.

#cmu-11868 #ai-course #cmu #fine-tuning #peft #lora #qlora #quantization

ai guideAug 25, 2026

The Complete Unsloth Guide: Fine-Tune and Run LLMs Locally, Faster

Unsloth is the fastest, most VRAM-efficient local LLM fine-tuning tool — 2× training speed and 70% less VRAM. In 2026 it added a Desktop app that bundles inference, training, image/video generation, web search, and agent integration into a complete local AI workstation.

#unsloth #llm #fine-tuning #local-inference #lora #qlora #gguf #self-hosted