Memory System

Agent Swarm agents aren't stateless. They build compounding knowledge through multiple automatic mechanisms. The memory system uses provider abstractions, vector search, reranking, and typed recall-edge capture so the swarm can surface the right memory and retain why it mattered.

How Memory Works

Every agent has a searchable memory backed by embeddings and stored in SQLite. The system is built on two provider abstractions:

• EmbeddingProvider — Converts text to vectors. Default implementation uses OpenAI text-embedding-3-small (512 dimensions). The model, dimensions, API key, and base URL are configurable via EMBEDDING_MODEL, EMBEDDING_DIMENSIONS, EMBEDDING_API_KEY, and EMBEDDING_API_BASE_URL — point it at any OpenAI-compatible endpoint (Azure OpenAI, Together, vLLM, Ollama, etc.). Swappable for other providers in code.
• MemoryStore — Persists and retrieves memories. Default implementation uses SQLite with sqlite-vec for KNN vector search. Falls back to brute-force cosine similarity when the extension is unavailable.

Embedding endpoints

Set the endpoint in the /setup memory step or in Settings → Integrations → Memory (embeddings). Both offer presets for OpenAI, OpenRouter, Vercel AI Gateway, Azure / Microsoft Foundry, and a custom URL. They send one test embedding and save the settings only when it works.

Azure / Microsoft Foundry. Azure serves its OpenAI-compatible v1 API under /openai/v1, and the model is the name of your embedding deployment:

EMBEDDING_API_BASE_URL=https://<resource>.services.ai.azure.com/openai/v1
EMBEDDING_MODEL=<embedding-deployment-name>
EMBEDDING_API_KEY=<resource-api-key>

The Azure preset takes the resource name or URL and appends /openai/v1. The deployment must honor the dimensions parameter (text-embedding-3-* does, text-embedding-ada-002 does not). A base URL without /openai/v1 fails with "endpoint path not found". The legacy /openai/deployments/<name>?api-version= form is not supported.

Memory Sources

Memories are automatically created from:

• Session summaries — At the end of each session, a lightweight model (default: DeepSeek v4.1 Flash via OpenRouter, pinned separately from the general default models; configurable through MEMORY_RATER_MODEL) extracts key learnings: mistakes made, patterns discovered, failed approaches, and codebase knowledge. These summaries become searchable memories. Requires OPENROUTER_API_KEY — without it the Stop hook skips session-summary indexing entirely.
• Task completions — Every completed (or failed) task's output is indexed. Failed tasks include notes about what went wrong, so the agent avoids repeating the same mistake.
• Explicit stores — Agents call the memory-store tool (or ctx.swarm.memory_store from a script) with a title, content, scope, and tags. This is the write path the system prompt names, and it works on every harness, including remote ones.
• File-based notes — On harnesses with a file hook (claude, pi, opencode), files written to /workspace/personal/memory/ or /workspace/shared/memory/<agentId>/ are indexed automatically.
• Lead-to-worker injection — The lead agent can push specific learnings into any worker's memory using the inject-learning tool, closing the feedback loop.

Memory Scopes

Automatic Scope Promotion

The inject-learning tool creates swarm-scoped memories by default, so learnings injected by the lead are available to all workers.

TTL & Expiry

Memories have a time-to-live (TTL) based on their source type. Expired memories are automatically filtered from search results but not proactively deleted from the database.

Expired memories can still be retrieved by ID via memory-get — only search results are filtered. The memory-delete tool provides explicit cleanup when needed.

Memory Retrieval & Reranking

Before starting each task, the runner automatically searches for relevant memories and includes them in the agent's context.

Search Process

1. Task description is used as the search query
2. An embedding is generated via the EmbeddingProvider
3. Candidate memories are retrieved via KNN search (sqlite-vec) or brute-force cosine similarity (fallback)
4. Reciprocal-rank fusion blends vector matches with a full-text pass before reranking by default; set MEMORY_HYBRID_SEARCH=0 to use vector-only retrieval. The fused score stays on the cosine's [0,1] scale: cosine × (0.5 + 0.5 × normalisedRRF), so rank 1 in both arms keeps the full cosine and rank 1 in one arm keeps 75%
5. The candidate set is expanded with 1-hop memory-link neighbors by default; set MEMORY_GRAPH_EXPANSION=0 to disable it (see Graph Expansion)
6. Candidates below the minimum similarity floor are dropped, then the survivors are reranked using a composite score
7. Top matches whose pre-boost relevance (rawSimilarity) exceeds 0.55 are included in the task context as "Relevant Past Knowledge". The gate reads relevance, not the composite score, because the reranker's multipliers can lift a weak match above any fixed composite threshold. Each memory_retrieval row records both similarity (composite) and relevance

Every retrieval row records its provenance (retrievalSource: vec, fts, hybrid, fallback, or graph), so the usefulness readout can compare how often each retrieval arm's results actually get cited.

Reranking

Raw vector similarity alone isn't enough — a curated manual memory from last month can still be more useful than a noisy session summary from yesterday. The reranker computes:

finalScore = similarity × sourceQuality × recencyDecay × accessBoost

Before reranking, candidates with raw cosine similarity below MEMORY_MIN_SIMILARITY (default 0.1) are discarded as noise. The remaining candidate set is fetched at 3× the requested limit, then narrowed after reranking. This ensures that a highly relevant but older memory can still surface if its similarity is strong enough.

Graph Expansion

By default, search results gain 1-hop graph neighbors: after the store returns its candidates and before reranking, the expander follows outgoing memory_link rows (resolved wikilinks pointing at live memories) from each candidate and injects the linked memories as additional candidates. A memory that is lexically and semantically distant from the query can therefore still surface because a strong hit links to it — "Auth flow gotchas, see [[deploy-checklist]]" pulls the checklist into results for an auth query. Set MEMORY_GRAPH_EXPANSION=0 or false to disable expansion.

Mechanics:

• Derived score — parentRawSimilarity × linkStrength × damping (damping 0.7), where parentRawSimilarity is the parent's [0,1] match score from its own arm, so neighbors rank below the hit that pulled them in. The parent's pre-decay similarity is used; the neighbor's own recency decay is applied exactly once by the reranker.
• Cap — at most 5 new candidates per search, selected by the reranker's composite score (not raw similarity), so stale low-value links can't crowd out better ones.
• ACL + filters — neighbors respect the caller's scope (agent/swarm/all), lead visibility, source filter, and expiry rules; expansion can never widen what a search is allowed to see.
• Provenance — expanded results carry retrievalSource: "graph", so the usefulness readout measures whether graph hits earn citations before the flag is enabled more widely.
• Fail-open — an expansion failure never poisons search; the original candidates are returned unchanged. With the flag off, results are byte-identical to no-expansion.

Both memory retrieval features retain explicit rollback flags so operators can compare per-arm citation data or restore the previous retrieval path without changing code.

Usefulness Readout

GET /api/memory/usefulness (query params days, threshold) answers "is memory useful?" from the measurement tables the raters populate:

• Volume — retrieval rows, distinct memories, and search/get event split in the window.
• Per-arm breakdown — retrievals and citation rate grouped by retrievalSource (search events only), so vec/fts/hybrid/graph can be compared head-to-head.
• Citation rate per memory-source — how often manual vs file_index vs task_completion vs session_summary memories get cited in task evidence (positive/ratings in [0,1], plus the signed avgSignal mean).
• Posterior movement — how many Beta-Binomial usefulness posteriors have moved off the prior, and how many sit above threshold.

The same data renders as the Usefulness panel on the dashboard's /memory page (summary tiles + per-source and per-arm charts).

Manual Search

Agents can search and manage memories using MCP tools:

• memory-store — Create a memory: content, name, scope (agent or swarm), optional tags, taskId, intent, and key (a logical path such as /longterm/facts/swarm-runtime/slug; a name that starts with /longterm/ is used as the key; a key under /longterm must start with /longterm/company-story, /longterm/entities/people, /longterm/entities/customers, /longterm/facts, /longterm/decisions, /longterm/workstreams or /longterm/timeline, and the memory then never expires and is protected from cleanup; /longterm/company-story, /longterm/entities and /longterm/timeline are lead-only). Long content is chunked on headings and embedded in the background. The owner is always the caller, for both scopes.
• memory-search — Search with natural language queries. Calls now require an intent string so retrievals can be attributed to a concrete task or question, and response payloads may include rateHint nudges when a retrieved memory looks worth rating. Pass keyPrefix (for example /longterm/facts/) to search one logical path only. The filter runs inside each retrieval arm and the graph expansion, before the result cut. Reranking multiplies the score by a path weight: /longterm/company-story 1.4, /longterm/entities 1.3, /longterm/facts 1.2, /longterm/decisions 1.1, /longterm/timeline 0.8, a /longterm/decisions doc tagged superseded 0.3. A memory with any key outside /longterm (every legacy memory) is inbox material and keeps its score.
• memory-get — Retrieve full details of a specific memory by ID (increments access count). Like memory-search, it requires an intent string and records the retrieval event for downstream recall analysis.
• memory-edit — Edit an existing memory in place without changing its ID. Agents may edit their own memories; editing another agent's memory by ID requires a lead, even when that memory is swarm-visible. Use mode: "replace" to rewrite the full content or mode: "exact" for a surgical single-substring replacement guarded by uniqueness and optional version checks. Pass newKey alone to move a memory to another logical path on every chunk without changing its ID, rating posterior, access count or author. Moving into /longterm also clears the expiry on every chunk; moving out does not bring it back.
• memory-delete — Delete a memory. Agents can delete their own agent-scoped memories; only the lead can delete swarm-scoped memories, even ones a worker wrote. DELETE /api/memory/{id} (used by the script SDK's memory_delete) enforces the same rule, and POST /api/memory/list returns an agent caller only its own memories plus swarm-scoped ones.
• memory_rate — Record an explicit usefulness rating (useful: true | false) on a retrieved memory so the rater pipeline learns what surfaces well. Optional referencesSource (free-form <source>:<identifier>, e.g. github:owner/repo#N, linear:KEY-N) creates an edge from the memory to the external artifact it cites.

Recall Edges & Memory Links

Search and retrieval events feed a lightweight memory graph:

• Intent-tagged retrieval rows capture why a memory was searched or opened, not just that it was touched.
• Typed memory links connect memories to other memories ([[wikilinks]] in content) and to external artifacts such as GitHub PRs, agent-fs paths, and other entity references resolved from memory content.
• Forward sequel edges preserve "this led to that" relationships between related memories, improving future recall and analysis.

The link graph has a full read side:

• Traversal — memory-get (and GET /api/memory/:id) returns the memory's outgoing links and inbound backlinks, ACL-filtered: linked-memory metadata is only included when the viewer may see the target, and hidden or deleted targets are indistinguishable from unresolved wikilinks.
• Search expansion — resolved memory-to-memory links power Graph Expansion when enabled.
• Self-maintenance — editing a memory prunes links derived from removed content (re-resolving links from the new content in one transaction) while preserving sequel edges, so the graph tracks what memories actually say. A wikilink that pointed at a not-yet-existing memory re-resolves on the next edit once the target exists.

Writing Memories

Store a memory as soon as something worth keeping is learned, with the memory-store tool:

memory-store({
  name: "API auth header needs the Bearer prefix",
  content: "The API requires the Bearer prefix on all auth headers. Without it you get a misleading 403 instead of 401.",
  scope: "agent",
  tags: ["example-api"],
})

The seeded memory skill carries the full guidance: what makes a good memory, when to pick swarm scope, how to check for a near-duplicate with the memory-dedup-check script before storing, and how to triage with memory-edit, memory-delete, and memory_rate. On harnesses with a file hook, a file written under /workspace/personal/memory/ is indexed too.

What to Save

• Solutions to problems you solved
• Codebase patterns you discovered
• Mistakes you made and how to avoid them
• Important configurations
• Instructions from the lead or user

What Not to Save

• Session-specific context (temporary state)
• Unverified conclusions
• Information that duplicates existing documentation

Memory Categories

When the lead injects learnings via inject-learning, they're categorized:

Configuration

Reranking parameters are tunable via environment variables:

Architecture

src/be/memory/
├── types.ts              # EmbeddingProvider + MemoryStore interfaces
├── constants.ts          # TTL defaults + reranking params (env-overridable)
├── reranker.ts           # Scoring: similarity × source quality × recency × access
├── index.ts              # Singleton getters
└── providers/
    ├── openai-embedding.ts   # OpenAI text-embedding-3-small
    └── sqlite-store.ts       # SQLite + sqlite-vec KNN search

The provider interfaces make it straightforward to add alternative implementations (e.g., a different embedding model or a Postgres-backed store) without changing any consumer code.

Related

• Hook System — The PostToolUse hook that auto-indexes memory files
• Agent Identity & Configuration — How identity files persist across sessions
• Architecture Overview — System-level view of how memory fits in
• Memory API Reference — REST API endpoints for searching and managing agent memories