MIT 6.7960 L08: Transformers — Tokens, Attention, Positional Codes, and How They Relate to MLPs/CNNs/GNNs
A Transformer is not an architecture from nowhere: tokens discretize data, attention does soft aggregation, positional codes restore order. Seen next to MLPs/CNNs/GNNs, all of them are special cases of 'weighted aggregation over neighbors'.