Signature
← Back to Overview

MAXIM

Semantic Memory

How Maxim Builds Lasting Knowledge from Experience

Episodic memory records what happened. Semantic memory captures what things mean. Humans don't remember the exact moment they learned that fire is hot — they just know it. That knowledge was distilled from many episodes into a concept. Maxim implements this same progression: repeated experiences get promoted from episodic memories into stable semantic concepts.

Three Memory Layers

Biological Inspiration

The brain separates memory into episodic (hippocampus — personal experiences), semantic (anterior temporal lobe — general knowledge), and procedural (cerebellum — motor skills). Damage to the ATL causes semantic dementia: patients can describe their wedding day (episodic) but can't explain what a "wedding" is (semantic). The two systems are anatomically distinct but deeply interconnected.

Maxim implements three memory layers, each handling a different kind of knowledge:

Hippocampus

Episodic Memory

"What happened." Rich contextual records of observe→decide→act→evaluate cycles.

  • Record type: EpisodicMemory
  • Decay: 7-day max without access
  • Compression: 2.5KB → ~200 bytes

ATL

Semantic Memory

"What things mean." Concepts, facts, and relationships stripped of episodic context.

  • Record type: SemanticMemory
  • Decay: 30-day max (slower than episodes)
  • Built from: Promoted episodes + agent inference

Angular Gyrus

Mathematical Memory

"What the numbers say." Facts, formulas, methods, and learned statistical patterns.

  • Record type: MathRecord
  • Categories: FACT, FORMULA, METHOD, CONSTANT, PATTERN
  • Seeded with mathematical constants at startup

The MemoryLayer Protocol

All three layers implement the same abstract protocol, enabling the MemoryHub and CrossLayerGraph to work with any layer uniformly:

MemoryLayer ABC class MemoryLayer(ABC): layer_name → str # "hippocampus", "atl", "angular_gyrus" store(record) → str # Store record, return ID get(record_id) → Record # Retrieve by ID remove(record_id) # Delete + cleanup edges recall(limit, **filters) # Filtered retrieval recall_associated(seed_ids) # Spreading activation recall graph → DependencyGraph # Internal associative graph save(path) / load(path) # Persistence consolidate(**kw) # Compress, decay, prune register_capture_callback(cb) # Notify on new record register_deletion_callback(cb) # Notify on removal stats() / __len__ / __iter__ # Inspection

All three layers share a common base record type (MemoryRecord) with fields for ID, timestamps, access tracking, and long-term status. Each layer extends this with domain-specific fields.

Layer Record Type Key Fields Compressed Form
Hippocampus EpisodicMemory perception, context, decision, action, outcome, run_id CompressedMemory
ATL SemanticMemory name, definition, category, properties, provenance, confidence CompressedSemantic
Angular Gyrus MathRecord name, category, domain, verbal, code, confidence CompressedMathRecord

ATL: Anterior Temporal Lobe

Biological Inspiration

The anterior temporal lobe is the brain's "semantic hub" — where concepts are stored independently of the episodes that formed them. You know that dogs bark, have four legs, and are pets, without remembering the specific experiences that taught you this. The ATL integrates information from multiple sensory modalities into amodal concept representations.

Maxim's ATL stores SemanticMemory objects — concepts with names, definitions, properties, and typed relationships to other concepts. It mirrors the Hippocampus architecture (context indexing, associative graph, consolidation) but with slower decay and higher stability.

SemanticMemory Record

Data Structure SemanticMemory(MemoryRecord): name: str # "coffee mug", "navigate_tool_reliability" definition: str # Natural language definition category: str # "object", "person", "action", "causal_pattern", "operational_pattern" properties: dict # {"color": "blue", "location": "kitchen"} provenance: enum # How this concept was formed source_episode_ids: list # Hippocampus episodes that contributed confidence: float # min(0.99, 0.5 + 0.1 × √reinforcement_count) reinforcement_count: int # How many episodes confirmed this embedding_text: str # Text used for EC similarity matching

Concept Provenance

Every concept knows how it was formed, enabling the system to weight newer, less-verified concepts differently from well-established ones:

EPISODIC_CONSOLIDATION

Extracted from repeated episodes via NAc reward signals. The most common path — "I've seen this pattern enough times to call it knowledge."

AGENT_INFERENCE

Proposed by the StatisticianAgent from confirmed operational patterns. "The data shows this is a real trend, not noise."

DIRECT_INGESTION

From RAG / document ingestion. Knowledge imported directly without episodic formation.

HYBRID

Multiple sources contributed. Both episodic observation and statistical confirmation.

ATL-Specific Features

  • find_or_create — Deduplication by name similarity. Before creating "coffee_mug", checks if "coffee mug" already exists. Returns (id, was_created) tuple.
  • recall_similar — Semantic similarity search via EC embeddings. "Do I already know about this?" Used by the promoter to avoid concept duplication.
  • Context indexing — O(1) lookup by name, category, or property key/value. "All concepts with category=object" is instant.
  • Consolidation — Same pattern as Hippocampus but with slower decay (30-day max age vs 7-day). Concepts are more stable than episodes.

Concept Memory

Biological Inspiration

Concepts in the brain aren't isolated labels — they're rich nodes linking back to the experiences that formed them. The concept "dog" connects to episodic memories of specific dogs, motor programs for petting, and statistical expectations about behavior. This multi-modal grounding is what gives concepts their meaning. Maxim's Concept class mirrors this: each concept maintains cross-layer references back to its source memories.

The Concept class extends SemanticMemory with a memory_refs dictionary that tracks which records in each memory layer are associated with this concept. This provides the backbone for concept extraction, grounding, pattern completion, and context building.

Concept Data Structure Concept(SemanticMemory): # Inherits: name, definition, category, properties, confidence, ... memory_refs: dict[str, dict[str, None]] # layer_name → {record_id: None, ...} # Example: memory_refs = { "hippocampus": {"ep-001": None, "ep-042": None, "ep-103": None}, "angular_gyrus": {"math-salience-mug": None}, } # Methods: add_ref(layer, record_id) # Link a memory to this concept remove_ref(layer, record_id) # Unlink a memory reinforce(source_id) # Increment confidence + access tracking

Why dict[str, None] Instead of set?

The memory_refs uses dict[str, None] (not set) as an ordered set. This guarantees true FIFO pruning when refs exceed MAX_REFS_PER_LAYER (200) — the oldest references are evicted first, preserving recency ordering without sorting overhead.

Capacity Enforcement

Concepts respect the same capacity management as other memory records:

  • Per-concept refs: Each layer's reference set is capped at 200 entries (FIFO eviction).
  • ATL capacity: ATL enforces max_concepts via the same MemoryStrategy pipeline used by Hippocampus — AccessBasedStrategy (30-day window) plus TemporalAwareStrategy when SCN is available.
  • Consolidation: ATL.consolidate() prunes low-confidence, stale concepts the same way Hippocampus prunes old episodes.

Concept Extraction

The ConceptExtractor runs as an async background worker that continuously transforms raw episodic memories into concept references. When Hippocampus captures a new episode, the extractor queues it for processing on a dedicated thread — never blocking the agent's perception-decision-action loop.

Extraction Pipeline Hippocampus.capture() │ ↓ (capture callback) ConceptExtractor._on_capture(episode) │ ↓ (enqueue, non-blocking) Worker Thread │ ├─ Extract object names "mug", "chair" → category=object ├─ Extract person names "Dennis" → category=person ├─ Tokenize goal string "navigate_to_kitchen" → ["navigate", "kitchen"] ├─ Tokenize tool name "grasp" → category=action │ ↓ (for each extracted term) ATL.find_or_create(name, category) │ ↓ concept.add_ref("hippocampus", episode.id) concept.reinforce(episode.id)

Token Normalization

Goal strings and tool names are split into meaningful tokens via normalize_tokens(). This handles underscores, camelCase, and common separators:

"navigate_to_kitchen"

→ ["navigate", "kitchen"]

"graspObject"

→ ["grasp", "object"]

Stop words ("to", "the", "a", "is", etc.) are filtered out. The extractor also handles compression callbacks — when an episode is compressed, its refs are migrated to the compressed record's ID.

Concept Grounding

Biological Inspiration

In neuroscience, "grounding" is the process of connecting abstract symbols to sensory experience. The concept "heavy" isn't just a word — it's grounded in the felt experience of lifting heavy things. The parietal cortex (IPS/Angular Gyrus) provides this mathematical grounding: converting raw observations into statistical summaries that give concepts quantitative meaning.

The ConceptGrounder transforms raw episodic references into statistical summaries stored in the Angular Gyrus. When a concept accumulates enough episode references, the grounder extracts numerical properties (salience, novelty, success rate) and runs them through the IPS→AG pipeline to produce MathMemory records.

Grounding Pipeline Concept (memory_refs: 8+ episodes) │ ↓ load episodes from hippocampus ConceptGrounder.ground_concept(concept, episodes) │ ├─ Extract salience values [0.8, 0.7, 0.9, ...] ├─ Extract novelty values [0.5, 0.3, 0.6, ...] ├─ Compute success rate 0.875 (7/8 episodes) │ ↓ IPS assessment IPS.assess(values) → {mean, std, is_random, ...} │ ↓ AG storage AngularGyrus.store(MathMemory) │ ↓ cross-layer linking concept.add_ref("angular_gyrus", math_record.id) CrossLayerGraph: QUANTIFIES edge (AG → ATL)

What Gets Grounded

Property Source AG Record Example Verbal
Salience perception.salience across episodes MathMemory "mug salience: mean=0.78, std=0.12"
Novelty perception.novelty across episodes MathMemory "mug novelty: mean=0.35, std=0.21"
Success Rate outcome.success across episodes MathMemory "mug success rate: 0.875 (7/8)"

Jaccard Co-occurrence Modulation

When two concepts frequently appear in the same episodes, the grounder strengthens the relationship between them via Jaccard similarity on their shared episode sets. If concepts A and B share 6 out of 10 episodes (J=0.6), the grounder creates or reinforces a RELATED_TO edge with weight proportional to co-occurrence.

Typed Relationships

Concepts in isolation aren't knowledge — knowledge is the relationships between concepts. The ATL uses a Semantics manager that wraps the DependencyGraph with typed, directional edges.

Pre-Registered Relationship Types

Type Symmetric? Example
IS_A No "coffee_mug" IS_A "container"
HAS_PART No "robot_arm" HAS_PART "gripper"
CAUSES No "grasp_action" CAUSES "object_held"
TRENDS_WITH Yes "navigate_failures" TRENDS_WITH "goal_latency"
CORRELATES_WITH Yes Confirmed by AG cross-metric correlation
PHASE_LOCKED_TO No "success_rate" PHASE_LOCKED_TO "circadian" (metadata: phase, coupling)
PREDICTS No "morning_activity" PREDICTS "high_success"

The RelationshipRegistry also supports runtime extension — agents can propose new relationship types during operation. The registry tracks whether each type is built-in or agent-proposed, and persists across sessions.

Cross-Layer Graph

Each memory layer has its own internal associative graph. The CrossLayerGraph connects records across layers, enabling queries like "starting from this episode, what concepts and math patterns are related?"

Cross-Layer Architecture Hippocampus ATL Angular Gyrus (episodes) (concepts) (math/patterns) ┌───────────┐ ┌───────────┐ ┌───────────┐ │ episode_1 ├─ASSOCIATES──┤ │ │ pattern_1 │ │ episode_2 ├─ASSOCIATES──┤ concept_A ├──IS_A───────┤ │ │ episode_3 │ │ concept_B ├──CAUSES─────┤ pattern_2 │ └─────┬─────┘ └─────┬─────┘ └─────┬─────┘ │ │ │ │ ╔══════════════════════╧══════════════════════╗ │ ├────║ CrossLayerGraph ║───┤ │ ║ ║ │ │ ║ DERIVED_FROM: ATL concept ← episodes ║ │ │ ║ INSTANCE_OF: episode → ATL concept ║ │ │ ║ STATISTICALLY_CONFIRMS: AG pattern → ATL ║ │ │ ║ QUANTIFIES: AG record → ATL concept ║ │ │ ║ TEMPORALLY_CORRELATES: any ↔ any ║ │ │ ║ COMPUTED_FROM: AG ← source data ║ │ │ ║ INFORMS: any → any ║ │ │ ╚═════════════════════════════════════════════╝ │ │ │ └──────────────────────────────────────────────────────┘

Cross-Layer Spreading Activation

The most powerful feature is cross-layer spreading activation. Starting from any record in any layer, activation spreads:

  1. Follow intra-layer edges via the layer's internal associative graph
  2. Follow cross-layer edges to records in other layers
  3. Recurse with exponential decay (default 0.5 per hop) until threshold or max depth
  4. Return activations grouped by layer — { "hippocampus": [(id, score)], "atl": [...], "angular_gyrus": [...] }

Example Traversal

Starting from a Hippocampus episode where tool_navigate failed:

  1. Intra-layer: Find similar failure episodes (Hippocampus associative graph)
  2. Cross-layer INSTANCE_OF → ATL concept "navigate_tool_reliability" (confidence=0.73)
  3. ATL IS_A → ATL concept "tool_capabilities"
  4. Cross-layer STATISTICALLY_CONFIRMS → AG PATTERN record (R²=0.73, slope=-0.02/day)
  5. Cross-layer TEMPORALLY_CORRELATES → AG record linking decline to circadian phase 0.8 (evening)

The Promotion Pipeline

Biological Inspiration

Memory consolidation during sleep transforms labile hippocampal traces into stable neocortical representations. The NAc (reward system) plays a gating role — rewarding experiences are preferentially consolidated. This is why emotionally significant events become lasting knowledge while mundane details fade.

The SemanticPromoter orchestrates the progression from episodic observations to stable semantic knowledge. It scans multiple PromotionSources for qualifying patterns, filters noise, and creates ATL concepts with full cross-layer traceability.

Promotion Pipeline PromotionSources IPS Gate SemanticPromoter ──────────────── ──────── ──────────────── NAc (reward patterns) ─┐ ├──→ IPS randomness ──→ Deduplicate (ATL recall_similar) StatisticianAgent ─┤ quality gate Extract recurring elements (confirmed patterns) │ (reject noise) Create/reinforce concept │ Form relationships Future sources... ─┘ Create cross-layer edges │ ↓ ATL SemanticMemory + CrossLayer edges + Relationship types

PromotionSource Protocol

Any system that detects patterns can be a promotion source. It just needs one method:

get_promotion_candidates(min_confidence=0.6, min_observations=3) → list[PromotionCandidate] # Each candidate provides: PromotionCandidate: pattern_name: str # "grasp → success", "stat:tool:navigate:success" category: str # "causal_pattern", "operational_pattern" confidence: float # Source system's confidence source_memory_ids: list[str] # Episodic IDs that contributed metadata: dict # Source-specific extras

NAc Path (Causal Patterns)

NAc tracks event→outcome links via Rescorla-Wagner learning. When a causal link reaches sufficient confidence and observation count, it becomes a promotion candidate.

Example: "grasp + coffee_mug → success" observed 5 times with confidence 0.8 → promote to ATL concept "grasping coffee mugs is reliable" with DERIVED_FROM edges to source episodes.

StatisticianAgent Path (Operational Patterns)

Confirmed patterns from the IPS→AG escalation pipeline become candidates. No episodic IDs needed — the AG MathRecord provides the evidence.

Example: "tool_navigate declining (R²=0.73)" → promote to ATL concept with provenance=AGENT_INFERENCE and STATISTICALLY_CONFIRMS cross-layer edge to the AG PATTERN record.

IPS Randomness Quality Gate

Not every NAc pattern deserves permanent semantic status. The IPS provides a lightweight noise filter before the heavyweight promotion machinery runs:

  • NAc candidates: IPS assesses the randomness of observation timing. If events occurred at random intervals (no temporal structure), they're likely coincidental — rejected.
  • StatisticianAgent candidates: Already passed IPS/AG assessment during the pattern detection FSM — just checks confidence ≥ 0.4 (no re-assessment needed).
  • Small samples (< 8 observations): Gate allows through conservatively — too few data points to assess randomness.

When Promotion Runs

Promotion is part of the mandatory consolidation cycle during MemoryHub.on_session_end():

Consolidation Ordering 1. SCN save (oscillator state + bin indices) 2. Promote (NAc + StatAgent patterns → ATL concepts, IPS filters noise) 3. Consolidate (compress, decay, prune — all 3 layers) 4. Persist (save all layers + cross-layer graph + EC embeddings + NAc) Ordering is critical: promotion runs BEFORE consolidation so source episodes still exist when cross-layer edges form.

Concept-Aware Recall

The ConceptContextBuilder provides real-time concept context during the agent's perception-decision loop. When the agent sees objects, people, or pursues a goal, the builder finds matching ATL concepts and enriches them with relationship data and AG statistics — all within a configurable time budget.

ConceptContextBuilder Flow Current Percept (detected_objects, detected_people, active_goal) │ ↓ normalize_tokens() for goals, direct lookup for objects/people _find_matching_concepts() → list[Concept] │ ↓ for each concept (sorted by confidence, capped at limit=5) ├─ _collect_relationships() ATL typed edges (IS_A, CAUSES, ...) ├─ _collect_layer_enrichment() AG MathMemory (mean, std, verbal) ├─ ground_concept() if grounder available + budget allows │ ↓ list[dict] → StructuredContext.concept_context

Budget-Bounded Grounding

AG grounding involves IPS statistical assessment, which can be expensive for concepts with many episodes. The builder enforces a time budget (default 50ms) and gracefully degrades:

Budget Available

Full context: relationships + AG stats + grounding results.

Budget Low

Relationships + cached AG records. No new grounding.

Budget Exhausted

Name + category + episode count only. Still useful for basic context.

Context Entry Format

What Each Entry Contains { "name": "mug", "category": "object", "confidence": 0.87, "episode_count": 12, "relationships": [ {"type": "IS_A", "target": "container", "weight": 0.8}, {"type": "RELATED_TO", "target": "kitchen", "weight": 0.7} ], "properties": { # From AG grounding (if budget allows) "mean_salience": 0.78, "success_rate": 0.92 }, "layer_context": [ # From AG MathMemory records {"layer": "angular_gyrus", "name": "mug_salience", "verbal": "mean=0.78, std=0.12", "confidence": 0.85} ] }

Pattern Completion

Biological Inspiration

When you see a friend reach for their coffee mug, your brain automatically predicts they're about to drink — before they do it. This is pattern completion: partial cues activate stored patterns, filling in the expected outcome. The hippocampus and neocortex collaborate: concepts activate associated episodes, which activate predicted actions and outcomes. Maxim's PatternCompleter implements exactly this chain.

The PatternCompleter predicts likely outcomes for a partially-formed episode by traversing the concept graph. When a new episode is forming (has perception + context, but no decision/action/outcome yet), the completer chains: matching concepts → linked episodes → past outcomes.

Pattern Completion Chain FORMING Episode (perception: ["mug"], goal: "grasp_mug") │ ↓ _find_matching_concepts() Concepts: [mug (object), grasp (action)] │ ↓ collect hippocampus refs (deduplicated) Episode IDs: {ep-001, ep-042, ep-103, ep-042, ...} → deduplicated │ ↓ load + sort by recency, cap at MAX_EPISODES=20 Past Episodes: [ep-103 (newest), ep-042, ep-001 (oldest)] │ ↓ extract outcomes PredictedOutcome[]: tool="grasp", success=True, goal="grasp_mug", confidence=0.9 tool="grasp", success=True, goal="grasp_mug", confidence=0.8 tool="look", success=True, goal="inspect", confidence=0.7 │ ↓ enrich with AG math context (memory_refs intersection) PredictedOutcome[0].math_context = [ MathContextEntry(name="mug_salience", verbal="mean=0.78", ...) ]

PredictedOutcome

Each prediction is a typed dataclass with full traceability back to the source episode and optional mathematical enrichment:

Typed Contract PredictedOutcome: tool: str # "grasp", "navigate", "look" success: bool # Did it work in the past? goal: str | None # What goal was being pursued? confidence: float # Source episode's decision confidence source_episode_id: str # Traceable back to specific memory math_context: list | None # AG enrichment (MathContextEntry[]) MathContextEntry: name: str # "mug_salience", "grasp_success_rate" verbal: str # "mean=0.78, std=0.12" confidence: float # Statistical confidence domain: str # "concept_grounding"

Wiring

PatternCompleter is created by MemoryHub during multi-layer wiring and connected to MemoryAgent via set_pattern_completion_fn(completer.complete). During the FORMING stage of memory formation, MemoryAgent calls the completion function to get predictions before the agent commits to a decision.

Episodic Predictions

From full EpisodicMemory records: uses decision.confidence, action.tool_name, outcome.success, and decision.intent for goal.

Compressed Predictions

From CompressedMemory records: uses the compressed fields directly. Confidence defaults to 0.3 (less certain than full episodic recall).

Knowledge in the Agent Loop

The point of all this is to make the agent smarter. Knowledge flows into the LLM's reasoning context via two channels in StructuredContext:

knowledge_context

Merged ATL concepts + AG patterns via cross-layer spreading activation. Broad, session-level knowledge.

Ranked by relevance, capped at 8 entries.

concept_context

Live concept context from ConceptContextBuilder. Percept-specific: only concepts matching current objects/people/goal.

Includes relationships, AG stats, budget-bounded grounding.

What the LLM Sees # knowledge_context (broad semantic + statistical knowledge) - "coffee_mug" (object, confidence=0.87): A ceramic container typically found in kitchens. Relationships: IS_A container, PROPERTY_OF kitchen_area - "stat:tool:navigate:success" (math:PATTERN, confidence=0.73): Navigate tool showing declining success rate (R²=0.73, slope=-0.02/day). # concept_context (percept-specific, with AG enrichment) - "mug" (object, confidence=0.87, 12 episodes): Relationships: IS_A container, RELATED_TO kitchen Stats: salience mean=0.78, success_rate=0.92 # pattern_predictions (from PatternCompleter, during FORMING) - Predicted: tool=grasp, success=True, confidence=0.9 Math: mug_salience mean=0.78, std=0.12

Together, these channels give the LLM a unified "what do I know about what I'm seeing?" view: broad knowledge context for general reasoning, live concept context for what's currently perceived, and pattern predictions for what's likely to happen next.

← Previous Math & Statistics Next → Attention & Salience

All Chapters

Overview The Agent Brain Memory Systems Math & Statistics Semantic Memory Attention & Salience Body Awareness Communication Usage Guide Technical Deep Dive Operating Modes Networking & Mesh Roadmap