Reference · STOR 323

Notation decoder

The math symbols the slides use, explained with real-world examples and pictures instead of code. Read once, come back whenever a formula looks like noise.

The hat: ˆ

Imagine you're guessing a stranger's weight just by looking at them. There's their actual weight (a real number, but you don't get to see it). And there's your guess, based on what you observed. The hat symbol is math's way of writing "my guess," so it never gets confused with "the real thing."

Same trick everywhere: β (the true, unknowable number) vs. β̂ (our estimate of it).

Subscripts: f^{^}_D

Picture two different weather forecasters. One trained their forecasting method on ten years of local rainfall data. The other used a different city's data. They'll come up with different guesses — different f^{^}'s. The little D underneath just labels which batch of data produced this particular guess, the same way you'd label two jars of homemade jam with the date they were made.

Different data in, different guess out. The subscript just keeps track of which data made which guess.

Arrow notation: D --A--> f^{^}_D

Think of baking a cake:

The arrow just means "goes through." D is your raw data (ingredients). A is the fixed process that never changes (the recipe steps — mix, pour, bake at 350°). What comes out — f^{^}_D — depends entirely on what ingredients you put in. Same recipe, different ingredients, different cake.

Set-builder notation & "model family" — the big one

This is the one that trips people up, so let's build it by hand instead of just reading it.

Real-world setup: you're pricing houses. You believe bigger houses cost more, and you believe the relationship is a straight line — some starting price, plus a bit more for every extra square foot. You don't know the exact numbers yet. So before looking at any data, you've already made a decision: "I will only ever consider straight-line pricing rules." That decision — "only straight lines allowed" — is the model family. It's the shape of every rule you're willing to consider, before you've picked the specific one.

Below, you are the machine. Drag the two sliders to move the red line until it passes as close as possible through all five real house sales (the dots). Watch the error number drop as you get closer.

What you just did by hand is exactly what a training algorithm does automatically, just faster and more precisely. Now here's what the notation is really saying:

So the "useful information" packed into the notation is: (1) what shape every candidate is allowed to have (a straight line, not a curve), (2) exactly which knobs are adjustable (here, just two: starting price and rate-of-increase), and (3) that "training" literally means searching those knobs for the setting with the least error — which is the game you just played.

Sum notation: Σ

Think of a grocery receipt — you add up every item to get the total. That's all Σ means: "add these all up."

Σi=1n x_i just means "add up x₁ through xₙ" — walk down the receipt and total it.

Argmax

Imagine a talent show where the audience claps for each contestant, and a meter measures how loud the applause is. Argmax doesn't tell you how loud the loudest clap was — it tells you who got it.

argmax of these four scores is Contestant C — the name/label of the winner, not the 0.72.

Probability notation: ℙ(Y = 1 | x)

Read the bar | as the word "given." "What's the chance of snow, given it's below freezing?" You're not asking about all days ever — you've already narrowed the world down to freezing days first, then asking about snow within just that narrower group.

ℙ(Y = 1 | x) = "the chance the label is 1, given that we've already observed input x" — same funnel, just with x as the narrowing condition instead of "below freezing."

Stacked conditions: FPR_g(t) = ℙ(δ_t(S) = 1 | Y = 0, G = g)

Story first, from the hospital screening slide. One hospital uses one screening rule — refer for follow-up if the test score is ≥ 0.30 — on everyone. It applies that exact same rule to two neighborhoods, Group A and Group B. Here's what actually happened to the people who were not sick:

Same test, same threshold, same rule for everybody. And yet: a healthy person in Group A is twice as likely to get a false alarm (40% vs. 20%) as a healthy person in Group B. Nobody changed the rule between groups — the disparity was hiding inside a single "false positive rate" number until you split it by group. That's the concrete thing. Now the label for it:

That four-stage funnel, read right to left in the formula, is exactly FPR_g(t) = ℙ(δ_t(S) = 1 | Y = 0, G = g):

• δ_t(S) = 1 — "the rule flagged this person" (the thing we're asking the chance of)
• | Y = 0, G = g — "given" two conditions stacked together: this person is actually healthy, and they're in group g. Commas inside the conditioning bar just mean "and."
• The subscript g on FPR_g is the same labeling trick as f^_D earlier — it just says which group this particular rate was computed for.

FNR_g(t) is the mirror image — same funnel, but starting from "given: actually sick (Y=1)" and asking the chance the rule missed them (δ_t(S) = 0). That's the "missed" bar in the charts above.

The sigmoid: σ(z)

Picture a dimmer switch that can only sit strictly between fully off and fully on, never quite touching either end. σ(z) = 1/(1+e−z) takes any real number — however extreme — and squashes it into that range, (0, 1), so it can be read as a probability. Feed it 0 and it always lands exactly in the middle: σ(0) = 0.5, "no lean either way," no matter what the underlying data looks like.

L(β) vs. L′(β) — how wrong, and which way to fix it

The step itself, β(1) = β(0) − η·L′(β(0)), is just "current value, minus a nudge in the direction the gradient says makes things worse." Subtracting a negative gradient means adding — that's why a negative L′(β) pushes β up, not down.

‖β‖₁ and ‖β‖₂² — two different ways to total up "how big"

Both take a list of coefficients and return one number describing their overall size — but they disagree on what "size" means. ‖β‖₁ = Σ|βj| just adds up the magnitudes, treating a single big coefficient and several small ones that add to the same total as equally "big." ‖β‖₂² = Σβj² squares each one first — so one coefficient of 2 (contributing 4) costs far more than four coefficients of 0.5 each (contributing 4×0.25 = 1 in total). That's the whole reason L1 and L2 regularization, built from these two norms, push coefficients toward different shapes: L1 doesn't mind concentrating size in one spot, L2 actively prefers spreading it out.