Skip to content
All tags

#self-supervised

2 posts
tech MIT 6.7960 導讀 (Fall 2024 OCW)

MIT 6.7960 L11: Representation Learning (Reconstruction-Based) — Autoencoders, VQ, Self-Supervision

Representation learning compresses raw data into a 'useful' vector: autoencoders force a meaningful latent space via reconstruction, VQ discretizes it into a codebook, and self-supervision turns 'mask-and-reconstruct' into free supervision.

#mit-67960#deep-learning#representation-learning#autoencoder#self-supervised#fall-2024
tech MIT 6.7960 導讀 (Fall 2024 OCW)

MIT 6.7960 L18: Transfer Learning — Pretraining, Feature Extraction, and Fine-Tuning Strategies

Transfer learning's core insight is 'features learned on big data are good general-purpose representations': freeze the backbone and train only a linear head when downstream data is tiny; full fine-tune when data is plentiful; reach for LoRA / adapter when compute is tight. SimCLR and MAE removed the need for upstream labels and pushed downstream quality another notch.

#mit-67960#deep-learning#transfer-learning#pretraining#fine-tuning#simclr#mae#self-supervised#fall-2024