Skip to content
All tags

#synthetic-data

3 posts
ai deep-dive Reading Stanford CS224N

CS224N Lecture 19: How Small Models Can Move Beyond Brute-Force Scaling

The final lecture frames Open Questions in NLP 2026 as smart scaling: prolonged RL, Prismatic synthetic data, RL as pretraining, and open collaboration seek reasoning gains beyond adding parameters.

#cs224n#reasoning#small-language-model#synthetic-data#stanford
ai deep-dive Reading Stanford CS224V

Stanford CS224V Lecture 14: Scaling Language Models When Data Is the Bottleneck

The final lecture is not a complete LLM-training tutorial. It studies data efficiency under fixed data and abundant compute, revisiting epochs, batches, ensembles, self-training, and conditions for synthetic continued pretraining.

#cs224v#stanford#language-model#data-efficiency#synthetic-data
ai deep-dive Reading Stanford CS336

CS336 Lecture 14: Filtering, Deduplication, and Mixing Turn Raw Web Data into Training Data

Lecture 14 moves raw documents through language, quality, and safety filtering; exact and near deduplication; and source mixing. Each stage reshapes model behavior, while synthetic instruction and agent trajectories extend the pipeline into executable environments.

#cs336#training-data#deduplication#synthetic-data#llm