Skip to content
All tags

#minimind

2 posts
aideep-dive從零訓練一個 LLM

MiniMind: Train an LLM From Scratch for $0.40

MiniMind is an open-source project for training LLMs from scratch: a 64M Dense model and a 198M-A64M MoE model that run the entire chain — Pretrain → SFT → LoRA → DPO → PPO/GRPO/CISPO → Agentic RL — in ~2 hours on a single RTX 3090 at roughly 3 RMB (~$0.40). Every core algorithm is implemented natively in PyTorch with no high-level wrappers.

#minimind#llm#training#open-source#pytorch#pre-training#sft#lora#dpo#grpo#moe#reinforcement-learning
aideep-dive從零訓練一個 LLM

Running MiniMind on RunPod: From Zero to a Chatting Model

The hands-on installment of the series: rent an RTX 3090 on RunPod (Secure Cloud $0.5/hr, Community Cloud $0.22/hr), follow the MiniMind README through pretrain (~1.21h) + SFT (~1.10h), spend roughly $0.55–1.50 USD total, and chat with your own 64M model trained from scratch in the terminal.

#minimind#runpod#llm#training#gpu#pytorch