Floors

Sugar Pill or Aspirin?

Maxim is a software project of mine: an AI agent with memory systems loosely modelled on parts of the brain, named after the hippocampus, the nucleus accumbens and others. The names are a way to organise the design. They are not evidence that it works, and nothing below leans on them.

Imagine you've invented a headache pill. There are two very different questions you can ask about it.

The first is whether the active ingredient does anything. You find out by giving some people the real pill and others an identical pill with the ingredient taken out. If only the real one helps, the ingredient is doing something. Researchers call this kind of test, where you remove one part and keep everything else, an ablation.

The second is whether your pill beats what's already on the shelf. You find out by testing it against plain aspirin. I call that kind of comparison a floor: the most basic, no-frills option that anything fancier has to beat to be worth the trouble.

A pill can pass the first test and fail the second. Plenty of ingredients beat a sugar pill and still aren't better than aspirin. Until October, Maxim had only ever taken the first test.

The Drowning Test

Maxim's best result comes from Minecraft. An agent is held underwater in a pool until its air runs low, which Maxim registers as pain. During this training it does nothing; it just goes through it. Then it's put back in the water, and each test is cut off just before the pain would start again. The question is whether it gets out on its own before it gets hurt.

The part doing the work is a simple piece of wiring. When pain arrives, Maxim attaches fear to whatever situation it's in. The next time it recognises that situation, the fear pushes it to act.

Beating the Sugar Pill

The first test was the sugar-pill kind. I ran the agent against an identical copy with the fear wiring disconnected: same dives, same pain. After training, the agent with fear got out before the pain on every run. The copy without it never did.

That result still stands, and it says something real: take the fear wiring out and the behaviour disappears. What it can't say is whether the wiring is special. Almost anything plugged into that spot would beat having nothing there.

Beating Aspirin

A floor is a plain agent that sees what Maxim sees, can do what Maxim can do, and gets the same rolls of the dice, but has none of the brain-inspired parts. Each floor answers a different sceptic.

Since 10 October the project has a strict rule: nothing new gets called earned unless it beats the floors that apply to it. Something that only ties is marked partial. That is still worth publishing. It just isn't a win.

The Result

I re-ran the fear agent alongside the coin-flipper and the trial-and-error learner, eight runs each, and all forty runs came back clean. The trial-and-error learner got ten practice dives in which it could actually try things. The score is whether each agent got out on its first test after training. Later tests are partly habit, because by then a successful escape has also been rewarded.

The fear agent clearly beat the coin-flipper: a statistical test says a gap that big is very unlikely to be luck (p = 0.0035). Against the trial-and-error learner there was no difference at all (p = 1.0). A tie doesn't prove the two are equally good, since eight runs each is too few for that. It only means I have no evidence the fear agent is better. On 14 October the drowning result went from earned to partial.

Who Decides When to Act?

The tie makes sense once you split getting out of the water into two questions: when to act, and what to do.

Back to the pills. Suppose your invention is a smart pill that learns the early warning signs of a headache from past ones and releases its medicine before the pain starts. That's really two things: a sensor that learned when, and a medicine somebody chose for it.

Maxim's fear is the sensor. It learned when from pain it went through without doing anything: being underwater picked up fear, and standing on the shore didn't. The what was chosen by me. Maxim has a built-in reflex that links fear to an action called escape_water, which I built for this experiment and named so the reflex would find it. A built-in choice like that is called a prior. One of the design reviewers summed it up: the fear wiring “learned when; the prior's name match supplied what.”

Now look at the comparison. To stop the plain learners acting blindly on dry land, the test only switched them on once they were in the water. In pill terms, a nurse handed over the aspirin the moment the headache began. That gave them the when for free, so the trial only compared the what, which is the half Maxim didn't learn either. The test plan said so in writing before any results came in.

The clearest number in the run makes the same point. The trial-and-error learner limited to the two escape-type actions got out on its very first time underwater, on 8 of 8 runs, with no experience at all. The built-in reflex carries the act.

Same Drug, Different Bottle

There is a third kind of comparison: the simplest textbook method that does the same job as your invention. Drug makers know this one well. If your new molecule turns out to work just like an old generic, your trial will tie with the generic, and that tells you something useful about what you've actually made.

For Maxim's fear, the twin is a standard algorithm that learns how good or bad each situation is from what tends to follow it. Given the same reflex, it is Maxim's fear wiring under another name, so the test plan expects a tie. That isn't embarrassing. It means any claim worth making lives somewhere else: in how Maxim recognises situations, in what the fear connects to, in what it does that the textbook version can't.

Two later experiments ran into exactly this. One showed the fear passing between separate agents: 12 of 12 agents that received another agent's memory, and had never felt the pain themselves, got out of the water. The other showed the fear carrying over to a second pool. Both rest on the same fear wiring, so on 15 October I marked both partial too, without re-running them, and wrote the decision down. What survives is narrower but real: a learned fear can be packaged up and moved between agents, and Maxim's sense of “underwater” carries over from one pool to another. Neither shows Maxim doing better than a plain learner would.

Faster Isn't the Same as Better

The fear agent needed no practice actions at all. The trial-and-error learner needed 31 tries in a typical run (the median of the eight), or 16 with only the two escape-type actions to choose from. That's a real gap, and in a world where every wrong try costs something, it would matter.

But the score was whether each one got out on the first test, and they tied. Learning fast and doing well in the end are different things, and winning one doesn't let you claim the other. That's why I fixed which one counted before the run. The fear agent's head start isn't all its own work either: its zero comes from a reflex that already knew the answer.

Rigging the Trial, Both Ways

The easiest way to beat a floor is to pick one that can't win, and the first draft of this test had one. It trained a trial-and-error learner on the same dives the fear agent got, where nothing the agent chooses actually happens. The pain arrives no matter what it picks, so its scorecard fills up with noise. A reviewer called it out as a floor that “can't fail to lose.” It was dropped before any data came in, and the real floor became a learner that actually gets to try things.

The trap has a quieter version. The learner's first settings made every untried action look better than one it had already tried, so in a practice simulation it kept exploring and got out on the first test only about 10% of the time. That floor was built to lose too. The settings were changed before the real run, and the change is on record. The plan also names the next version of the trap in advance: giving a plain learner such a fine-grained view of the world that it can't tell two nearly identical situations are the same. That would rig the trial in Maxim's favour, and the reviewers are told to refuse it.

It cuts the other way as well. A floor that can't lose, because it's handed the answer or the task leaves nothing to learn, turns every result into a tie and tells you nothing. In drug terms, you learn nothing from a trial against expired aspirin, or against a sugar pill that's secretly aspirin. The honest floor is one you think might win.

What Comes Next

Here is where Maxim stands. The drowning fear beats chance. It doesn't beat a plain learner that's told when to act. The fear's learned sense of when still beats the copy with the wiring cut. The two follow-up experiments are partial. The chatbot floors haven't run, so nothing new on the chatbot side counts as earned.

The next experiments run the textbook twin as a competitor too, so every result says how much it adds over the standard version of itself. And to beat a floor, a test needs a situation where a sensible plain learner could actually lose. The best candidate is recognising something new as similar to something familiar. Maxim files each new situation by how much it resembles ones it has already seen, so a fear learned in one place can fire somewhere it has never been. A plain scorecard treats that new place as brand new and has to learn it from scratch. If Maxim's brain-inspired design is earning its keep, that's where it should show. If the plain learner ties there too, it goes on the record the same way this one did.

Beating a sugar pill only tells you the ingredient does something. Whether the pill is worth taking is a question you have to put to something else.