YuLan-Mini: Squeezing a 2.4B Flagship-Scale Small Model Out of 1.08T Tokens
YuLan-Mini is a 2.4B open-source model from Renmin University's AI Box lab, trained on 48 A800 GPUs with only 1.08T tokens — scoring 37.8 on MATH-500 and 64.0 on HumanEval, beating Qwen2/Qwen2.5 peer models trained on 7T–18T tokens at math and code. What's public isn't a slogan: per-phase data mixes, pre-annealing optimizer states, and even W&B logs of the ablation studies.