The video loads from YouTube only after you press Play.
Transformers, the tech behind LLMs | Deep Learning Chapter 5
A 27-minute video by 3Blue1Brown about machine learning, on the AI shelf. A commenter in r/agi recommended it. It is pitched at intermediate level.
3Blue1Brown, Video, 27 min, 2024, on the AI shelf
Visualizing how data flows through transformer language models.
Study notes
Intermediate, How it works inside. Step 8 of 11 in From probability to modern AI
Helpful first
- Vectors and matrices
- Neural-network basics
What it builds
- Follow the broad steps of next-word generation.
- Represent words as vectors in an embedding space.
- Distinguish the model’s vocabulary from its learned representations.
Useful moments
- 3:08How a chatbot generates a word
- 3:45Similar meanings in embedding space
- 12:48The model’s vocabulary
Check yourself: Why does a language model need more than a dictionary lookup for each word?
Notes are based on metadata, Reddit context and selected English caption passages. They are not a review of every claim in the video.
