CS336 Lecture 14: Filtering, Deduplication, and Mixing Turn Raw Web Data into Training Data
Lecture 14 moves raw documents through language, quality, and safety filtering; exact and near deduplication; and source mixing. Each stage reshapes model behavior, while synthetic instruction and agent trajectories extend the pipeline into executable environments.