Lesson 0002
Last time: every model is a finger, never the moon. This time — there's more than one kind of finger, and choosing among them is itself a discipline.
In the Lotus Sūtra, the Buddha is described teaching the same truth through many different methods — parables, similes, gradual instruction — chosen not arbitrarily but because each audience, each moment, called for a different vehicle to the same understanding. This is upāya: skillful means. The teaching that awakens one student can leave another untouched, not because the truth changed, but because the method wasn't matched to the situation.
STOR 323 makes the identical move, in its own vocabulary: "output space first, model family second." Before reaching for linear regression, softmax regression, or k-nearest-neighbors, you have to know what shape a correct answer even takes. Reach for the wrong vehicle — treat an unordered category as if it were a number on a scale — and you get nonsense, the same way a parable told to the wrong audience lands as noise instead of insight.
Recall samskara from the last lesson — experience compressed into a lasting impression. That compression is one way of holding what's been learned. It isn't the only way.
Neither is superior in the abstract — this is the same upāya logic as before. A teaching compressed into principle travels light and generalizes; direct memory of particulars stays faithful to nuance a principle might smooth over. STOR 323 calls this split parametric vs. nonparametric. You now already know it by two other names.
When STOR 323 needs to choose among several unordered categories — which digit, which condition — it uses softmax regression. Each category gets a raw score, and softmax converts the scores into probabilities. The mechanism is worth sitting with:
A raw softmax score means nothing by itself — "3.4" isn't a probability, isn't even meaningfully large or small until it's placed against the other scores. Softmax's normalization (dividing by the sum of all the exponentials) is a mechanical enactment of that same insight: every category's probability is defined only in relation to every other category's. Push one score up, and every other probability shifts — not because anything about them changed, but because their meaning was never independent to begin with.
k-nearest-neighbors makes no claim about a general rule at all. Asked to classify a new point, it looks at the K nearest existing points and takes a vote. Its entire method is: judge this by what surrounds it.
kNN doesn't ask "what is the general rule for this category of thing?" It asks "who is nearby, and what are they?" — the same wager satsaṅga makes about people: that you are legible through your neighbors, not despite them. Try it below.
| Model family | How it holds what it learned | Its upāya |
|---|---|---|
| Linear regression | jñāna — a few coefficients | The answer is one number on a continuous scale |
| Softmax regression | jñāna — one coefficient set per category | The answer is one of several unordered, mutually-defining categories |
| k-nearest-neighbors | anubhava — the data itself, kept whole | The answer is best read through immediate company |
Answer from memory before checking.
Formal side: An Introduction to Statistical Learning, Ch. 4, on multi-class classification and kNN. Philosophical side: the Lotus Sūtra's parable of the burning house is the classic upāya text if you want the fuller story behind "skillful means"; the dependent origination and satsaṅga links above are worth reading in full. See RESOURCES.md.