Gesture Drive

The hard part isn't the tracking

MediaPipe locates 21 hand landmarks per frame, and that's Google's model — I haven't tried to improve on it. The work is everything after: raw landmark coordinates jitter. Feed the fingertip x straight into a steering value and the car shakes constantly even with your hand perfectly still.

Why not just smooth it?

Because the obvious fix trades one problem for another. Enough exponential smoothing to kill the jitter also adds enough lag that steering feels like driving a boat. The two requirements fight each other at any fixed cutoff.

The One Euro filter makes the cutoff a function of speed: heavy smoothing when your hand is still, almost none when you're moving deliberately — because during a fast motion you can't perceive the noise but you absolutely perceive the delay.

Its two parameters were measured, not guessed. Sweeping both against a still hand and a deliberate sweep, and comparing each setting against an EMA tuned to the same jitter reduction — the only fair comparison — gave:

My first attempt used beta 0.035 — small enough that the cutoff barely moved with speed. It cut jitter fine and tracked worse than the EMA it was supposed to beat: all of the cost, none of the point. The test caught it.

Two more details that decide whether it feels right

• Calibration by median, not mean. Your neutral position is wherever you happen to hold your hand, so it's recorded over 45 frames. The median, because one frame where the detector jumps would drag a mean far enough to bias every input afterwards.
• A dead zone that doesn't steal range. A hand held "still" wanders by a percent or two, so small offsets map to zero. But the remaining range is rescaled, or the dead zone would quietly make full lock unreachable.

The simulation has no rendering in it

step(state, dt, steer) is a pure function — no canvas, no DOM, no clock. That's what makes the parts that are awkward to verify by playing testable at a fixed timestep: that difficulty actually rises, that collision is symmetric, that a huge dt from a backgrounded tab can't tunnel the car through an obstacle, that a fixed seed replays identically. 33 checks, run in CI.