Back to Home
Inside an LLM
How a Transformer Reads Text
A Transformer reads all words together and looks for relationships between them not word by word.
This is a simplified learning demo, not the exact math used inside real Transformers.
Interactive Playground
What's Happening
Press Read Text to tokenize your sentence, then tap any word to see its relationships.
Live Visualization
Tokens will appear here. Tap a word to see how it relates to the rest of the sentence.
Statistics
Total Tokens
Selected Token
Strongest Rel.
Rel. Count
All Together
Reading Mode
How It Works
📝
Sentence
→
🔤
Tokens
→
👁️
Read All
→
🔗
Relate
→
🧠
Meaning
🎓
Beginner Tips
2 tips
Position still matters even without left-to-right scanning, transformers add positional encodings so "dog bites man" ≠ "man bites dog".
Every word attends to every other word at once, unlike older RNN models which read one word at a time. This parallelism makes transformers fast to train.
💡
Key Takeaway
A Transformer does not read text like a simple left-to-right scanner. It looks across the sentence to understand how words relate.
A pronoun like "it" is understood by checking which nouns appeared earlier the model sees the whole sentence at once.