Back to Home
Inside an LLM
Attention
Attention is how a Transformer decides which words matter most when understanding another word.
This is a simplified learning demo, not the exact attention used inside real language models.
You just learned that a Transformer looks at relationships between words. Attention is the method it uses to decide which relationships are most important.
Interactive Playground
Simple learning attention
Press "Analyze Sentence" to begin
3
Nearby focusWide focus
Live Visualization
Attention Graph
Strong attention
Weak attention
Attention Scores
Focus:
Click a word above to see attention scores
Statistics
Total Words
Focus Word
3
Attention Spread
Strongest Attention
Weakest Attention
How It Works
📝
Sentence
→
🔤
Tokens
→
👆
Focus Word
→
📐
Calc Attention
→
✨
Highlight
🎓
Beginner Tips
2 tips
Attention ≠ reading order. AI doesn't scan left-to-right. Click a word and watch it connect to distant words that shape its meaning.
Real models use multi-head attention, many attention patterns run simultaneously, each focusing on different aspects like grammar, context, and coreference.
💡
Key Takeaway
Attention helps the model focus on the most useful words instead of treating every word equally.