Skip to content
All tags

#olmo

2 posts
aideep-dive從零訓練一個 LLM

How fully transparent LLMs are built: OLMo 3's model flow and LLM360 K2's 360-degree openness

"Open-source LLM" is a spectrum: weights-only (Llama), weights plus data (most fully open projects), or data order, intermediate checkpoints, and training logs all released (LLM360 K2, OLMo 3's model flow). This piece unpacks the two projects that pushed transparency furthest: OLMo 3 shipped the first fully open 32B thinking model in November 2025, and K2 is the first 65B-class model whose checkpoints even include optimizer states.

#olmo#llm360#llm#open-source#pre-training#training-data
aideep-dive

OLMo: The Only Language Model Family That Open-Sources Its Training Data

Allen AI's OLMo is the only language model family that fully publishes weights, training data (Dolma, 9.3T tokens), training code, all intermediate checkpoints, and evaluation tools. OLMo 3's 32B Think model hits 96.1% on MATH — and you can use OlmoTrace to trace any output back to the exact training data that produced it.

#llm#open-source#olmo#allen-ai#training-data#dolma