Bloom Scroll

Information, error and trust

How do we measure information, correct it when it breaks and trust it without a referee?

7 videos, about 2 h 51 min. Watch in order, or jump to a timestamp.

  1. 1

    Introductory, Derivation, 32 min

    Reinventing Entropy | Compression is Intelligence Part 1

    • Connect Shannon’s questions with modern machine learning.
    • Ask what cross-entropy actually measures.
    • Construct a way to measure information using streams of bits.

    Jump to 0:50 shannon’s mathematics and modern ml, 1:03 cross-entropy in next-token prediction

  2. 2

    Introductory, Visual intuition, 17 min

    Information Entropy: The Math Behind Surprise

    • Assign a numerical value to surprise.
    • Treat information as surprise.
    • Understand entropy as an average over possible outcomes.

    Jump to 0:20 building the entropy machine, 0:31 information is surprise

  3. 3

    Introductory, Derivation, 20 min

    But what are Hamming codes? The origin of error correction

    • Recognize the inefficiency of simply copying every bit.
    • Use parity-check outcomes to distinguish error locations.
    • Understand what an error-correcting code can and cannot identify.

    Jump to 0:50 redundancy by repeating every bit, 5:38 introducing parity checks

  4. 4

    Intermediate, Derivation, 17 min

    Hamming codes part 2: The one-line implementation

    • Use multiple parity checks to binary-search for an error.
    • Interpret the result as the error position written in binary.
    • Connect the construction to a short implementation.

    Jump to 0:21 parity checks as a binary search, 4:29 spelling the error position in binary

  5. 5

    Intermediate, Built from scratch, 37 min

    What is error correction? Hamming codes in hardware

    • Treat extra bits as parity checks rather than copies.
    • Compute parity with exclusive-or gates.
    • Connect error correction to physical circuitry.

    Jump to 0:20 correcting a flipped bit, 1:43 extra bits as parity checks

  6. 6

    Intermediate, How it works inside, 23 min

    How Does SHA-256 Work?

    • Break a hash computation into primitive operations.
    • Track constants and message words through the computation.
    • Separate understanding the steps from proving cryptographic security.

    Jump to 5:01 shift-right and other basic operations, 10:59 mixing constants

  7. 7

    Introductory, Derivation, 25 min

    But how does bitcoin actually work?

    • Separate a transaction record from a trusted central authority.
    • Use cryptography to verify claims without trusting a person.
    • Understand why the ledger with the most computational work wins.

    Jump to 2:20 trustless verification using cryptography, 11:38 trusting the ledger with the most work