CS224N Lecture 16: Hallucination, Creativity, Work, and Alignment

🌏 中文版

The official CS224N Winter 2026 schedule places lecture 16 on February 26, 2026, but does not name a lecturer; this article therefore attributes it only to the course staff. The official deck, AI's impact on humanity, has four agenda sections: hallucination, the paradox of AI-assisted creativity, workforce impact, and value alignment.

Why models hallucinate

Language Models (Mostly) Know What They Know studies self-evaluation signals, while Why Language Models Hallucinate analyzes training and evaluation incentives. A language model is trained to produce a next token even under uncertainty: it may estimate that it lacks an answer yet still guess because post-training rewarded confident responses. Stronger reasoning or higher overall accuracy does not automatically mean lower hallucination.

An engineering defense cannot be only “do not fabricate.” External claims require retrieval and checks that citations open and support the sentence. High-risk actions require human confirmation. Model confidence is a signal, not a substitute for verification.

The paradox of AI-assisted creativity

Generative tools can raise individual output or help people complete work, while aggregate output becomes more alike. If many creators use the same model, preference data, and default prompts, local improvement can create a population-level diversity tax.

Creativity evaluation therefore has at least two levels: quality of each work and diversity across a set. An average rating alone misses contraction of the creative space.

Work is not a binary “will the job disappear?” question

The deck shifts attention from occupations to tasks. Automation may replace some steps, accelerate others, and create new work in verification, integration, and responsibility. Average productivity gains do not imply equal distribution; skill, bargaining power, and error cost shape who benefits.

A useful analysis maps tasks, AI intervention points, remaining human judgment, and failure responsibility rather than citing one occupational exposure score.

Why value alignment is difficult

Human values are plural, conflicting, and context-dependent. Compressing preference into one reward discards minority views and non-tradeable constraints. A model can also overfit annotators, cultures, or deployment settings.

Alignment is ongoing governance rather than a completed training stage: who sets rules, how decisions are appealed, how out-of-distribution failures are monitored, and which authority remains human.

Separate hallucination failure types

Distinguish factual, citation, grounded-faithfulness, and tool-result fabrication. Each needs different evidence and risk depends on use.

Calibration and selective prediction

Use reliability and coverage–risk curves. Generated confidence is elicited, and post-training can reward confidence without correctness.

Why stronger reasoning need not be more honest

More generation can improve answers and add plausible false detail. Evaluate answer, support, and citations separately.

A citation-verification pipeline

Parse, resolve, locate supporting spans, classify support, and preserve original bibliographies/diffs. Resolve all citations and deeply audit semantic support.

Individual and population creativity

Randomize assistance and measure individual quality/productivity separately from population diversity, with traditional-tool baselines and expertise slices.

Mechanisms of a diversity tax

Shared models, preferences, interfaces, and anchoring concentrate output. Evaluate quality–diversity frontiers rather than rewarding noise.

Consent and attribution

Training consent, style imitation, attribution, and creator appeals are governance questions not solved by similarity scores.

Decompose workforce effects by task

Map exposure, reliability, error cost, physical/legal authority, augmentation, automation, and new verification work.

Productivity measurement traps

Include rework, severe errors, review burden, learning, field duration, and distributional effects—not only one-session speed.

Deskilling and the supervision paradox

Rare failures demand skills routine automation erodes. Preserve practice, evidence, progressive authority, and incident learning.

Distribution and power

Audit who receives gains, bears failures, is monitored, can refuse, sees logs, and can appeal across language and job-quality groups.

Value pluralism

Separate non-tradeable rights and authorization from learned preferences; majority rankings do not represent every group.

Reward-model governance

Version rubrics, annotators, disagreement handling, policy changes, rollback, labor conditions, and incident feedback.

Alignment evaluation

Test normal, edge, adversarial, shifted, multi-turn, and tool cases; measure over- and under-refusal; combine red teams with natural prevalence.

A pre-deployment worksheet

Document affected groups, authority, evidence, appeal, monitoring, rollback, abstention, human verification, approvals, diversity, and rework. Re-evaluate after drift.

Material gap

Winter 2026 recordings are not public. This article covers all four official agenda sections. Cases cited in the deck illustrate mechanisms and are not extrapolated into field-wide rates.

References

• Official CS224N Winter 2026 course page
• Lecture 16 AI's Impact on Humanity slides
• Language Models (Mostly) Know What They Know
• Why Language Models Hallucinate