Skip to content
All tags

#self-attention

2 posts
ai deep-dive 認識 AI 模型

Transformers and Attention: How Models Decide Which Words to Look At

The core of the Transformer is self-attention: for each token, the model computes how relevant every other token is, then takes a weighted sum. This lets the model reach across distance to figure out that 'it' refers to 'cat' not 'mat' — and is the foundation for how it handles long documents.

#transformer#attention#self-attention#nlp#ai-model#architecture
ai deep-dive Reading Stanford CS224N

CS224N Lecture 5: From Recurrence to the Transformer

Lecture 5 moves from the long-range and sequential bottlenecks of RNNs to self-attention and the Transformer. It shortens information paths and enables parallel computation, at the price of quadratic attention and separately encoded position.

#cs224n#transformer#self-attention#nlp#stanford