The video loads from YouTube only after you press Play.
Reinventing Entropy | Compression is Intelligence Part 1
A 32-minute video by 3Blue1Brown about information and entropy, on the Statistics shelf. It came up in 2 Reddit threads, in r/3Blue1Brown. It is pitched at introductory level.
3Blue1Brown, Video, 32 min, 2026, on the Statistics shelf
Reconstructs information entropy from the practical problem of compressing language.
Study notes
Introductory, Derivation. Step 1 of 7 in Information, error and trust
Helpful first
- Basic probability
- Binary numbers
What it builds
- Connect Shannon’s questions with modern machine learning.
- Ask what cross-entropy actually measures.
- Construct a way to measure information using streams of bits.
Useful moments
- 0:50Shannon’s mathematics and modern ML
- 1:03Cross-entropy in next-token prediction
- 4:08Sending data as a stream of bits
Check yourself: Why should a very common symbol get a shorter code than a rare one?
Notes are based on metadata, Reddit context and selected English caption passages. They are not a review of every claim in the video.
Reconstructs information entropy from the practical problem of compressing language.
