Expression Reader

Loss and held-out accuracy, per epoch early stopping keeps the best-validating weights, not the last ones

Confusion matrix where the mistakes actually go — more informative than the accuracy above

Turn the camera on, then hold each face for a couple of seconds. Forty frames per expression is plenty. It retrains automatically once two classes have samples.

The features are already good

MediaPipe returns 52 blendshape scores per frame — how open the jaw is, how raised the left brow is, how far each mouth corner has pulled. That is a well-engineered 52-dimensional feature vector arriving at 30 frames a second, and it turns expression recognition from a computer vision problem into a plain supervised classification problem.

So the model can be small — and hand-written

A multinomial logistic regression, trained with Adam, written from scratch. No TensorFlow.js, no autograd. The gradient of cross-entropy through a softmax is (p − y), which is one line, and the whole thing trains in well under a second on a few hundred samples.

• Features are standardised first. Blendshapes are all nominally 0–1 but their real spread differs by an order of magnitude — jawOpen swings across most of its range while noseSneerLeft barely moves. Unscaled, one step size cannot suit both and training oscillates.
• The softmax is max-shifted. Without it, exp() of a confident logit overflows to Infinity and every probability comes back NaN — a bug that hides well, because argmax over NaNs still returns something.
• Early stopping keeps the best weights. Not the final ones. Otherwise the accuracy you are shown is whatever the last epoch happened to land on.
• The bias is not regularised. Penalising it just biases the model away from the true class priors.

Why there is a rule baseline

The baseline is a hand-written mapping — smile blendshapes mean happy, brow down means angry — and it is labelled as rules everywhere it appears. Its job is to be the thing the trained model has to beat. Without a baseline, "84% accurate" is a number with no scale attached to it.

It refuses to learn from noise

Trained on 240 random vectors with random labels, the classifier reaches 42.8% on the training split and 31.7% held out — against a 33% chance baseline. It memorises the training data and generalises not at all, which is exactly what it should do, and is why the held-out split exists. There is a test asserting it.