Skip to content
All tags

#synthetic-data

3 posts
aideep-diveReading Stanford CS224N Natural Language Processing with Deep Learning

How Small Models Can Move Beyond Brute-Force Scaling | Stanford CS224N Lecture 19

The final lecture frames Open Questions in NLP 2026 as smart scaling: prolonged RL, Prismatic synthetic data, RL as pretraining, and open collaboration seek reasoning gains beyond adding parameters.

#cs224n#reasoning#small-language-model#synthetic-data#stanford
aideep-diveReading Stanford CS224V Agentic AI

Scaling Language Models When Data Is the Bottleneck | Stanford CS224V Lecture 14

The final lecture is not a complete LLM-training tutorial. It studies data efficiency under fixed data and abundant compute, revisiting epochs, batches, ensembles, self-training, and conditions for synthetic continued pretraining.

#cs224v#stanford#language-model#data-efficiency#synthetic-data
aideep-diveReading Stanford CS336 Language Modeling from Scratch

Filtering, Deduplication, and Mixing Turn Raw Web Data into Training Data | Stanford CS336 Lecture 14

Lecture 14 moves raw documents through language, quality, and safety filtering; exact and near deduplication; and source mixing. Each stage reshapes model behavior, while synthetic instruction and agent trajectories extend the pipeline into executable environments.

#cs336#training-data#deduplication#synthetic-data#llm