Imagination

The Concept

When the agent encounters a novel entity mentioned in percept text that has no existing SEM component, the imagination system designs one in real-time. The result is a fully functional entity with sensors, affordances, and failure modes — registered ephemerally for the session as an observable scene entity the agent can sense.

The Pipeline

Each stage is a gate. If any gate fails, the pipeline short-circuits gracefully — the agent falls back to verbal-only interaction with the entity. The trigger fires each turn after the agent has updated its state on the percept, before it decides what to do.

Entity Extraction

Lightweight NLP heuristics extract entity-like noun phrases from narration text. No external model required — this runs on pure string processing.

Two strategies work in parallel:

• Sentence-level intro patterns — catches "You see a rusty gate", "A massive golem blocks the path", "There is a glowing orb"
• Head-noun scanning — matches against a curated indicator vocabulary of entity-like words (weapon, creature, door, chest, etc.)

ComponentIndex: Two-Layer Lookup

Before imagining anything, each candidate phrase is checked against the ComponentIndex to see if an existing component already covers it:

If either layer finds a match, imagination is skipped for that phrase. This prevents duplicate components under different names. The catalog it searches is documented in the component library reference.

Gates

Three gates prevent imagination from firing at inappropriate times:

The thresholds are configuration; see the imagination reference.

EntityDesigner

When all gates pass, the EntityDesigner makes a single LLM call to generate a complete SEM component specification from the entity phrase and surrounding narrative context. It first infers a structural archetype (humanoid, creature, weapon…) and merges that archetype's body-part vocabulary into the generated spec, which cuts down on hallucinated modulator names.

Quick Validation

The spec is validated against the SEM protocol: required fields present, sensor ranges valid, modulator params typed, failure triggers well-formed. Invalid specs are discarded — the agent falls back to verbal interaction. The Asset Foundry is the batch equivalent, with gauntlet testing.

Bookkeeping

Imagined entities live in a separate overlay from the persistent component registry. During the session the entity is visible to lookups, added to the ComponentIndex for dedup, and registered as an observe-only scene entity: the agent can sense it but cannot call its affordances directly as tools. It interacts using its own body’s tools, while the imagined affordance names are encoded through the substrate path for concept transfer (EC → ATL → NAc eligibility) — the same lifecycle as pre-authored scene entities, described in the tools reference. A session-scoped cache shared by the agent and the orchestrator stores successes and failures alike, so a phrase both extract fires one design call, and a failed validation for "vague smoke" won't retry every turn.

At session end the overlay and cache are cleared, but the learning is not thrown away. Every episode and causal link from an imagined entity carries imagined=True provenance, and on discard those links get a 50% confidence decay. The agent retains partial learning ("chandeliers can collapse") with appropriately reduced confidence compared to verified real-world interactions; if a later encounter confirms the pattern, confidence rebuilds through standard Rescorla-Wagner updates.

Integration with Bio-Systems

Imagined entities participate in the full bio-pipeline, just like pre-authored ones: