MAXIM
Sound Orienting
The substrate's first real-hardware sensorimotor policy — learned, not programmed
What Happened
Core Result
A Reachy Mini learned to turn toward speech — which way and how far — starting from a completely empty reward substrate, with no LLM anywhere in the action path. Direction went from chance to 100% correct within ~10 hardware trials. It was already correct at trial 0 of the next session (loaded from the persisted substrate). Two independently trained substrates merged into a policy at least as good as either — the seed of fleet learning. And with the state representation fixed, it reached 100% on step size too: near the centre it learned that a big turn is actively harmful — from a single punished attempt it never repeated.
Nobody wrote a “turn toward the sound” rule. The robot's reward substrate (the nucleus accumbens model) discovered which turn direction reduces its bearing error in each situation, purely from experiencing relief when the error shrank — the same drive-pain-reduction credit that powers Maxim's interoceptive drives.
The Learning Loop
mic array (XVF3800 DoA chip) # localization is on-chip; Maxim does NOT learn to localize
↓ azimuth ∈ [-1, +1] # 0 = centered; the "centeredness" homeostatic drive's set-point
↓ state = azimuth bin # far_left / near_left / near_right / far_right
NAc.recommend_action(state) # substrate-primary: cluster-keyed reward bias, no LLM
↓ turn_left / turn_right # body-yaw step (the head's ±17° clamp made body the orient axis)
robot turns → chip re-measures # the world IS the model — physics closes the loop
↓ relief = |az₀| − |az₁| # drive-pain REDUCTION: positive = progress, negative = regress
NAc.update_cluster_reward(state, action, relief)
↓ persist # the learned policy survives restarts — and can be sharedThe design splits cleanly: the chip answers “where is the sound?” (that's solved hardware); the substrate learns “what do I do about it?” — which action, in which state, reduces the error. Credit is relief (error reduction per step), not raw pain: an earlier simulation study showed pain alone cannot drive action selection — it punishes correct-but-incomplete steps mid-approach — while dense per-step relief learns a perfect policy from every state.
Learned — Provably Not a Servo
A hard-coded servo would also drive the error to zero — and would be 100% correct from trial 1. So a working orient loop proves nothing by itself. Three pre-registered separations make the learning claim (full data: Exp 45 in the repo):
1. Learning curve
A frozen-policy probe (pure substrate lookup, no motion, no exploration) ran every 5 trials. From an empty substrate: 0% → 75% → 100% correct by trial 10, held through 40. In the magnitude run the executed-action rate went 0.29 → 1.00 — it started below the coin-flip baseline, meaning the untrained substrate was actively wrong, and converged to perfect. A servo can't have a learning curve, and certainly can't start out worse than chance.
2. Cross-session transfer
A new session loaded the persisted substrate and probed 100% correct at trial 0 — before any new experience. All 31 greedy actions that session turned the right way.
3. Merge across learners
Two independently trained substrates merged (nac_merge) into a policy at least as correct as either input — in rehearsal the merge even repaired one input's weak state. Fleet learning in miniature.
One moment from the live run says it all: on trial 2 an exploratory wrong turn made the error worse (negative relief); on trial 3, the very next visit to that state, the greedy choice had already flipped to the correct turn. One-trial operant learning, on hardware.
What the Hardware Taught Us
The first learning run half-worked, then poisoned itself. Chasing that produced a day of wrong answers and one lesson worth more than the result.
A wrong actuation assumption looks exactly like a broken sensor
We commanded the robot's base to turn and assumed the microphones — which live in the head — turned with it. They didn't. The robot's firmware holds the head's orientation fixed in world space unless you explicitly tell it otherwise, so the head counter-rotated and the mic array rotated barely a third of what we commanded.
Reading a nearly-stationary sensor while believing it moved produces a perfect imitation of a broken sensor: readings that lag, compress, and vary run to run. We generated six plausible explanations — a slow tracking estimator, settle lag, gear backlash, motor under-travel, speech-density effects, step-size limits — and every one was falsifiable, tested, and false. The actual answer was one sentence in the vendor's documentation, published the whole time.
One keyword fixed it. The sensor turned out to be excellent: linear to R² = 0.998 across ±80°, converging in 0.23 s, with no hysteresis and none of the pathologies we'd “measured.” The lesson now sits in the project's engineering rules: when a measurement disagrees with your model, verify that the thing you're sensing with actually moved — before theorizing about the sensor.
This also explains a curious asymmetry. Direction learning survived the bug entirely — it's sign-based, and a mic array that rotated only a third of the commanded amount still rotated the correct way. Magnitude learning was destroyed by it — it depends on overshoot, a threshold, and thresholds are exactly what a proportional error corrupts. The harder claim was the one that noticed the bug.
The Design Constants Are Derived, Not Chosen
With an honest sensor, magnitude learning still stalled at 75%. The instinct was “use finer bins.” That was wrong — and the right answer is a small piece of theory that transfers to any robot.
The state bins and the action set are duals
A turn of size d moves the error by S = d × gain. So a step takes error e to |e − S|, which means one action beats another exactly when the current error is nearer to its shift. The optimal policy is just nearest-neighbour quantization of the error onto your available step sizes — and the boundaries fall at the midpoints:
boundary = gain × (d_small + d_big) / 2 # Reachy: 0.546 × 1.2/2 = 0.33Our bins split at 0.5 — an arbitrary number that straddled 0.33. A bin spanning the boundary contains two opposite correct answers: it hands the learner consistently contradictory evidence and caps it, no matter how much exploration or reward shaping you add. Moving the boundary to the derived value took magnitude from 0.75 to 1.00.
So it was never about resolution — it was about alignment. And because the boundary depends on the robot's own measured gain, every robot derives its own. You cannot copy ours.
Running This on Your Robot
The learning core is robot-agnostic by construction — it's keyed on normalized azimuth bins and action names, never on hardware. Per the capability-driven embodiment principle, a new platform declares what it has; the cognition code doesn't change. Your robot supplies exactly four things:
1. A gated bearing source
Any front-end producing a normalized azimuth (−1..+1, 0 = front) plus a validity gate (speech detection, VAD, energy). On-chip DoA, a ROS audio stack, an external array — the robot's business. Never fabricate a direction.
2. A discrete yaw primitive
“Turn the sensing frame by ±step.” Which joint is the robot's choice (Reachy: base turntable; a biped: torso or step-turn) — but it must rotate the microphones, or the loop never closes.
3. A body declaration
A SEM body YAML with the azimuth sensor, the centeredness drive (set-point 0, zero drift — the world re-measures), and the two orient affordances. Copy the Reachy pattern.
4. Calibration answers
Sign (does a “left” turn move azimuth + or −?), the sensor's response curve (memoryless or tracker?), settle timing, step size vs noise, axis limits. All discovered by the standing protocol: primitives-smoke → sign self-check → sweep.
The full contract — including Atlas-class notes (external localization front-end, expensive step-turns, mics-must-turn-with-the-frame) and the apparatus-generated trial protocol that makes runs hands-off and balanced — lives in docs/embodiment/porting_orient_loop.md in the repo.
Sharing What Your Robot Learned
The learned policy is eight numbers with provenance — and zero personal data: substrate bundles carry distilled learning (reward biases, concept prototypes), never episodes, by construction. That makes a trained orient policy the first real payload for the Hivemind's substrate-sharing layer:
- Gauntlet — the frozen-policy probe validates any contribution in milliseconds, no hardware: a flipped-calibration or poisoned policy simply fails. This is promotion gate #1 of the planned Queen-tier trust topology (curated substrate flows out freely as signed releases; contributions earn their way in through validation).
- Merge —
nac_mergecombines contributors' reward biases (provenance-tagged, clamped), and the tool refuses to bundle any merge that isn't at least as correct as its inputs. - Bundle — the gauntlet-passed result exports as a domain-tagged substrate zip (“queen-mind” release). Another robot imports, merges, and starts its first session already knowing how to orient — identical hardware means the policy transfers by construction.
Where This Goes
Orienting is Layer 1 of a deliberate ladder, and it is now complete — direction and magnitude, on real hardware, learned from relief alone. Next rungs: ears (pinna-like shell structures + a raw two-mic binaural front-end to break the linear array's front/back ambiguity — turning a hardware blind spot into a learnable state, and now with a genuinely clean sensor baseline to measure against); vision on the same backbone (a camera bearing feeds the same centeredness drive; category recognition decides what's worth orienting to); and the north star — spatial co-location as a grounding substrate: when the head centers on something seen, heard, and named at the same pose and moment, that shared location becomes the index that binds sound, sight, and words.