Skip to content
All tags

#sft

3 posts
ai deep-dive 認識 AI 模型

Pre-training, SFT, RLHF: Three Stages That Turn a Text Predictor into a Useful Assistant

Every LLM goes through three training stages: pre-training reads the internet to learn language, SFT uses example conversations to learn the format, and RLHF uses human preferences to learn what a good answer looks like. The gap between a base model and a chat model is what the last two stages do.

#pre-training#sft#rlhf#dpo#fine-tuning#ai-model#training#alignment
ai deep-dive Reading Stanford CS336

CS336 Lecture 15: SFT Teaches Imitation; RLHF Begins Direct Preference Optimization

Lecture 15 divides post-training into imitation and optimization. SFT extracts pretrained capabilities from instruction-response data; RLHF uses pairwise feedback to bridge demonstrations and preferences. PPO and DPO both inherit data bias, reward overoptimization, and mode collapse.

#cs336#post-training#sft#rlhf#dpo
ai deep-dive Reading Stanford CS229

Large Language Models: Tokenization, Transformers, MoE, and SFT

Chapter 17 runs from next-token loss through Transformers, KV caches, MoE, and SFT, connecting an LLM's objective and architecture to its inference costs.

#cs229#llm#transformer#mixture-of-experts#sft