Harvard CS50 AI Synthesis (1): From Search to Language — The Complete Arc of Seven Weeks

🌏 中文版

⚠️ Version note: Lecture videos are Spring 2020 recordings (Weeks 0–5) and 2023 re-record (Week 6); project specs, distribution code, and check50 slugs follow the 2026 OCW site.

TL;DR

Seven weeks aren't isolated topics but a deliberate knowledge arc: deterministic search → logical reasoning → probabilistic uncertainty → combinatorial optimization → supervised/reinforcement learning → neural networks → language models. Each week's core abstraction builds on the previous; projects precisely target theoretical focal points.

The Complete Knowledge Arc Map

Week 0: Search          ──►  State space, goal-directed, optimal paths
         │
         ▼
Week 1: Knowledge       ──►  Symbolic representation, deductive inference, model checking
         │
         ▼
Week 2: Uncertainty     ──►  Probabilistic graphical models, Bayesian inference, stochastic processes
         │
         ▼
Week 3: Optimization    ──►  Local search, constraint satisfaction, taming combinatorial explosion
         │
         ▼
Week 4: Learning        ──►  Induction from data, trial-and-error, function approximation
         │
         ▼
Week 5: Neural Networks ──►  Distributed representation, backpropagation, deep abstraction
         │
         ▼
Week 6: Language        ──►  Sequence modeling, attention, pre-Transformer era

Three-Stage Evolution Logic

Stage 1: Symbolism & Deterministic Reasoning (Weeks 0–1)

Core Question: In fully observable, deterministic environments, how to find optimal solutions?

Week	Core Abstraction	Key Insight
Week 0 Search	State Space Graph + Frontier Strategy	Search strategy = data structure choice; Minimax reduces adversarial play to tree search
Week 1 Knowledge	Propositional Logic + Model Checking/Resolution	World = symbols + rules; Inference = syntax manipulation preserving semantics

Design Philosophy: AI = Search + Knowledge Representation. Degrees (BFS graph search) and Tic-Tac-Toe (Minimax tree search) validate uninformed/informed and adversarial search. Knights (model checking logic puzzles) and Minesweeper (Sentence KB inference) validate symbolic reasoning systems.

Stage 2: Uncertainty & Optimization (Weeks 2–3)

Core Question: Real world is partially observable, noisy, combinatorially explosive — how to decide?

Week	Core Abstraction	Key Insight
Week 2 Uncertainty	Bayesian Networks + Markov Models	Conditional independence = graph structure; Inference = variable elimination/sampling; PageRank = stationary distribution
Week 3 Optimization	CSP + Local Search/Annealing	Constraint propagation shrinks domains; Heuristics guide backtracking; Annealing accepts worse moves to escape local optima

Design Philosophy: Shift from "exact inference" to "approximate inference & search". Heredity (likelihood weighting) validates Bayesian net approximate inference; PageRank (iteration/sampling) validates Markov chain stationary distribution; Crossword (AC-3 + MRV/LCV backtracking) validates complete CSP solving pipeline.

Stage 3: From Data Learning to Representation Learning (Weeks 4–6)

Core Question: No hand-crafted rules — learn function mappings directly from data.

Week	Core Abstraction	Key Insight
Week 4 Learning	Supervised Classification + RL MDP/Q-learning	k-NN/SVM memorize/boundary; Q-learning learns value function from experience
Week 5 Neural Networks	Backpropagation + CNN/Distributed Representation	Chain rule computes gradients efficiently; Convolution captures translation invariance
Week 6 Language	N-gram/TF-IDF + Attention/Transformer	Statistical LM → Neural LM; Attention enables dynamic context

Design Philosophy: "Feature Engineering" → "Representation Learning" → "Attention Mechanism". Shopping (k-NN hand-crafted features) and Nim (Q-table tabular RL) demonstrate traditional ML/RL; Traffic (CNN end-to-end feature learning) demonstrates representation learning; Parser (CFG symbolic grammar) and Questions (TF-IDF statistical retrieval) demonstrate symbolic/statistical NLP, paving the way for Attention.

Core Abstraction Dependency Chain

Search Problem 5 Elements (Week 0)
    │
    ├─► State, Actions, Transition, Goal, Cost
    │
    ▼
Knowledge Base + Inference Rules (Week 1)
    │
    ├─► Symbols, Connectives, Models, Entailment
    │
    ▼
Probabilistic Graphical Models (Week 2)
    │
    ├─► Random Variables, Conditional Independence, Joint Distribution
    │     │
    │     └─► Exact Inference Exponential → Approximate Sampling
    │
    ▼
Constraint Satisfaction Problems (Week 3)
    │
    ├─► Variables, Domains, Constraints → Arc Consistency Shrinks Domains
    │     │
    │     └─► Backtracking + Heuristics = Practical Solver
    │
    ▼
Supervised Learning: Hypothesis Space Search (Week 4)
    │
    ├─► k-NN: Instance-based, Non-parametric
    ├─► SVM: Max-margin, Kernel Trick
    │
    ▼
Reinforcement Learning: Sequential Decisions (Week 4)
    │
    ├─► MDP: State, Action, Reward, Transition, Discount
    └─► Q-learning: Model-free, Off-policy, Convergence
    │
    ▼
Neural Networks: Differentiable Function Approximators (Week 5)
    │
    ├─► Backprop = Chain Rule Auto-diff
    ├─► CNN = Parameter Sharing + Local Receptive Fields
    │
    ▼
Language Models: Sequence Conditional Probability (Week 6)
    │
    ├─► N-gram: Markov Assumption, Count Statistics
    ├─► TF-IDF: Term Importance Weighting
    └─► Attention: Dynamic Context, Parallel Computation

Project Design Maps Theoretical Foci

Project	Week	Core Algorithm	Pedagogical Purpose
Degrees	0	BFS	Graph shortest path, Frontier abstraction
Tic-Tac-Toe	0	Minimax + αβ	Adversarial tree search, pruning optimization
Knights	1	Model Checking	Logic puzzles → Symbolic reasoning
Minesweeper	1	Sentence + Inference	Dynamic KB, Subset inference
Heredity	2	Likelihood Weighting	Bayesian net approximate inference
PageRank	2	Power Iteration / Sampling	Markov chain stationary distribution
Crossword	3	AC-3 + Backtracking	Complete CSP solving pipeline
Shopping	4	k-NN + Standardization	Supervised classification baseline, feature preprocessing
Nim	4	Q-learning	Tabular RL, Exploration/Exploitation
Traffic	5	CNN + Keras	End-to-end representation learning, Image classification
Parser	6	CYK + Generation	CFG syntax parsing, Ambiguity handling
Questions	6	TF-IDF + Cosine	Statistical retrieval, QA pipeline

2020/2023 Recordings in 2026 Context

Topic	Recording Vintage	Still Core in 2026	Evolved/Missing
Search/Logic	2020	DFS/BFS/A*/Minimax/Logic Inference	—
Probability/Optimization	2020	Bayesian Nets/Markov/AC-3/Annealing	Variational Inference, Advanced MCMC
Supervised/RL	2020	k-NN/SVM/Q-learning Basics	Deep RL, Off-policy Evaluation
Neural Networks	2020	Backprop/CNN/Optimizers	Transformers, ViT, Diffusion, LLM Fine-tuning
Language	2023 Re-record	N-gram/TF-IDF/Attention Basics	LLMs, RAG, Agents, Instruction Tuning, RLHF

Conclusion: First five weeks' "Classical AI Core" remains essential foundation in 2026; Week 6's Attention intro, though late, provides minimum gateway to modern LLM architectures. Gaps are in application layer (Prompt Engineering, RAG, Agents, Evaluation) — require separate study.

Suggested Learning Path

1. Strict Order: Week 0 → 1 → 2 → 3 → 4 → 5 → 6 (hard dependencies)
2. Projects in Parallel: Do project immediately after each week's lecture, don't batch
3. Math Reinforcement:
   • Weeks 0–1: Discrete Math (Graph Theory, Logic)
   • Week 2: Probability (Bayes, Markov)
   • Week 3: Combinatorial Optimization, Heuristics
   • Weeks 4–5: Linear Algebra, Calculus (Gradients), Convex Optimization
   • Week 6: Information Theory, Statistical NLP
4. Modern Follow-up: After CS50 AI, continue with:
   • Deep Learning with Python (Chollet) for Keras practice
   • Dive into Deep Learning for Transformer/LLM full architecture
   • Hugging Face Course for Fine-tuning, RAG, Agents

Series Links

• Overview (order 0)
• Week 0 Search (order 1)
• Week 1 Knowledge (order 2)
• Week 2 Uncertainty (order 3)
• Week 3 Optimization (order 4)
• Week 4 Learning (order 5)
• Week 5 Neural Networks (order 6)
• Week 6 Language (order 7)
• This Post: Synthesis 1 (order 8)
• Synthesis 2: Project Portfolio Comparison (order 9)
• Wrap-up: Timeless vs Changed, Next Steps (order 10)

References

• CS50 AI OpenCourseWare Main Site — All weeks, projects, videos, specs
• CS50 AI YouTube Playlist
• Weekly Notes (2020 ed. Weeks 0–5, 2023 ed. Week 6)
• Project Spec Pages & check50 Slugs (ai50/projects/2024/x/...)
• On this site: Global AI/CS Course Map — A3 Tier Definition
• On this site: Harvard AI/ML Course Map — CSCI S-80 Version Mapping