Skip to content
All tags

#transfer-learning

2 posts
tech MIT 6.7960 導讀 (Fall 2024 OCW)

MIT 6.7960 L18: Transfer Learning — Pretraining, Feature Extraction, and Fine-Tuning Strategies

Transfer learning's core insight is 'features learned on big data are good general-purpose representations': freeze the backbone and train only a linear head when downstream data is tiny; full fine-tune when data is plentiful; reach for LoRA / adapter when compute is tight. SimCLR and MAE removed the need for upstream labels and pushed downstream quality another notch.

#mit-67960#deep-learning#transfer-learning#pretraining#fine-tuning#simclr#mae#self-supervised#fall-2024
ai deep-dive Reading CMU 07-280

CMU 07-280 Lecture 15: Separating Pretraining, Transfer Learning, and Fine-Tuning

Lecture 15 splits a pretrained model into representation g and task head h: freeze g and train only the head, or fine-tune some or all parameters at a smaller learning rate depending on data volume and source-target distance.

#cmu#ai-course#transfer-learning#fine-tuning#representation-learning