Lesson 0015 · Homework 1, Problem 4

Regularization can prefer a shortcut

Sage already has two fitted candidates for flagging walkways. Same features, same task — but the penalty she adds decides which one ships, before either touches a new photo.

Problem 4(a)–(b)

Two candidates. One penalty. Different verdicts.

Candidate A concentrates weight on two features and zeroes the rest. Candidate B spreads the same total weight across all four. ‖β‖₁ can't tell them apart — ‖β‖₂² can.

Do not mix these up

Add the sizes. Or square them first.

‖β‖₁ — add up the sizes

‖β‖₂² — square first, then add

How this dataset is built

A photo becomes training data once a human confirms it.

Sage's four features come straight off a walkway photo — nothing about whether the walkway actually needed review. That part is added afterward, once a reviewer checks.

One photo, just collected

Obstruction shapeirregular

Surface contrastlow

Camera-border patternpresent

Time-of-day indicatordawn

Needs review?not confirmed yet

After a reviewer checks

Same four featuresx_i

Reviewer confirmsy_i = 1

Meaningflagged for review

A new photo, scored by the chosen model

Same four featuresx_i

Model usedwhichever penalty won

Decisionflag, or don't

Why this matters: the fitted coefficients only ever saw the four features — never the camera's maintenance schedule, never who happened to walk the storm-damaged route that week. If one feature quietly encodes something other than obstruction, the model can't tell the difference from inside the fit.

Problem 4(a)–(b), worked

Same total size. Different penalty. Different winner.

A = (1.50, 0, 1.50, 0), loss 0.30. B = (0.75, 0.75, 0.75, 0.75), loss 0.34. λ = 0.05.

Norms

‖A‖₁3.0 ‖A‖₂²4.5 ‖B‖₁3.0 ‖B‖₂²2.25

‖β‖₁ ties exactly. ‖β‖₂² does not — B's spread-out weight costs less once squared.

Regularized objective, λ = 0.05

A + L10.45 B + L10.49 A + L20.525 B + L20.4525

L1: A wins (tie on penalty, so raw loss decides).
L2: B wins (the penalty itself flips the verdict).

Problem 4(c)

A zero coefficient is a fact about this fit — not about the world.

Problem 4(d)

One camera was mostly used after storms.

The camera-border feature isn't measuring obstruction — it's measuring which camera took the photo, which happens to correlate with storm damage. An overall accuracy number can't see this.

storm-camera photos other-camera photos mixed together

Overall accuracy only one number, both cameras pooled the shortcut stays invisible

Split by camera, then check storm-camera performance vs. other-camera performance a sharp gap exposes the shortcut

Second check — intervene directly: swap or blank out the border pattern across otherwise-identical photos. If the prediction changes, the model was leaning on the border, not the obstruction.

What stays fixed?

Fixedthe four features, both candidates' raw fit Changeswhich penalty you add, which candidate wins Audita zero coefficient is a fit fact, not a world fact

Stuck on the norm arithmetic, the diamond/circle geometry, or the camera audit? Ask your teaching agent to walk through one panel.