Lesson 0010 · Capstone
Every idea from lessons 0005–0009, in the order it actually happened — including the part where the field's funding collapsed for over a decade because of four points on a grid.
Nothing here is new material. It's the same ideas — perceptron, XOR, MLP, attention, scale — with the people, dates, and accidents put back in. Read it after the technical lessons, not before: the names land differently once you already know what problem each person was actually solving.
A neurophysiologist and a homeless teenage logician co-wrote the first math model of a neuron — "all-or-none" firing as a logic gate. Pitts had been sleeping in libraries when McCulloch took him in. Their model is the ancestor of every perceptron in Lesson 0005.
Built the idea into real hardware — the Mark I Perceptron, a wall of motorized potentiometers "learning" to sort punch cards. The Navy funded it; the New York Times called it the "embryo of an electronic computer" that would soon walk, talk, and reproduce itself.
Their book Perceptrons didn't just note that XOR was hard — it proved no single perceptron, trained any way you like, can ever solve it (the exact case from Lesson 0005's "sharpen your pencil" exercise). The proof was airtight enough that funding for neural-network research collapsed for over a decade — the first "AI winter." The wall wasn't a bug in Rosenblatt's training method. It was baked into the shape of the model, and it took the field ten years to back up and stack instead of walking away.
Popularized backpropagation — the answer to the question that had stalled stacking for 17 years: when a deep network is wrong, which of the thousands of interior weights deserves the blame? Rosenblatt's rule could only blame weights wired directly to the output. Backprop passes blame backward through the layers via the chain rule — which is also why the hard step had to give way to smooth activations, since blame flows along slopes and a step has none. The AI winter thawed because someone worked out how to train the fix (the multi-layer perceptron), not because the idea of stacking was new — Minsky's own proof hadn't ruled out stacking, only single layers.
"Attention Is All You Need" — a title borrowed straight from a Beatles lyric. It was built to speed up machine translation. Nobody on the paper claimed the attention mechanism it introduced would become the backbone of every major chatbot within six years.
The training task is almost embarrassingly simple — guess the next word. Scaled up on enough text, that simple task turned out to produce something that could write code and pass exams — the surprise Lesson 0009 unpacks. The surprise wasn't the architecture; it was how far "just predict the next word" went once the data got big enough.
Every "sudden" breakthrough here was actually a decade-old idea meeting a missing piece — a training method, a compute budget, a dataset — that finally showed up. None of these people saw the whole arc coming from where they stood.