Decoupling the Control Plane of Hypatia: Applying MemGPT, RAPTOR, and Harness-1 to Multi-Agent Pipelines

Large language models have massive context windows, but passing an entire document corpus into a prompt chain creates two critical problems:

• Lost in the middle: Even if a model can accept 100,000 tokens, doing so creates “Lost-in-the-middle” attention degradation. As demonstrated by Liu et al. in “Lost in the Middle: How Language Models Use Long Contexts,” LLMs retain information at the absolute beginning and end of a long prompt but misplace or fail to recall facts buried in the middle of the text payload.
• Cost & Latency: Token costs scale linearly with prompt size, meaning a bloated context window passing sequentially across multiple agents incurs very API expenses and high execution delays.

RAG attempts to fix these problems.

• RAG slices that document into small fragments and retrieves only the top K chunks. This slashes prompt overhead and hence reduces cost.
• RAG partially solves lost in the middle problem by pruning away irrelevant text and presenting the LLM with a condensed payload of only K matching snippets, RAG keeps the prompt clean, allowing the model’s localized attention mechanisms to focus on the exact text provided.

However, the traditional RAG has few limitations

• It is flat: Traditional RAG breaks a document into disjointed chunks and pulls the top K matches based on proximity. It fails to perform well at global, multi-hop questions (e.g., “What is the overarching methodology?”) because it loses the big picture.
• It is externally dictated: In normal RAG, an external Python or Node script initiates the search. Before the LLM even sees the query, a script converts the input into an embedding, hits a vector database, grabs the top K chunks, and glues them into the prompt. The LLM has zero say in what data it receives.

How Recent Research Solves This

Below recent papers provide the architectural framework to solve these bottlenecks:

• MemGPT (Virtual Memory Management): This paper proposes treating the LLM’s context window like a CPU’s RAM. Instead of loading everything into the prompt, the agent is given tools to dynamically page-in specific raw blocks from an external storage ledger (disk) only when necessary.
• RAPTOR (Hierarchical Tree Indexing): Instead of chopping documents into flat chunks, RAPTOR recursively clusters and summarizes data from the bottom up. This creates a structured knowledge tree of granular local summaries (leaves) and broad global summaries (roots).
• Harness-1 (Stateful Cognitive Offloading): Traditional agents are trained over an append-only text transcript, meaning the model must repeatedly reread its own sprawling history just to remember what it has checked. Harness-1 proved that shifting the administrative burden -tracking history, verified claims, and candidate references -into an environment-side data harness allows the agent to execute purely stateless, high-level semantic logic.

Re-Architecting Hypatia’s Control Plane

This synthesized architecture from the above 3 papers was applied to Hypatia, an open-source autonomous multi-agent system designed to read, summarize, and deep dive scientific literature.

The Original Architecture

At its core, Hypatia is a multi-agent pipeline built on the Google Antigravity SDK, utilizing a specialized cluster of LLMs (Scout, Analyst, Explainer, Summarizer, Deep-Dive, and Critic) to sequentially ingest and evaluate scientific papers.

However, in its original architecture, the pipeline relied on stateful prompt chaining. For example, the summarization workflow was sequential and flat: an Analyst agent would read a raw PDF string, generate facts, and pass those facts and the raw text to a Summarizer. The Summarizer would then pass that entire accumulated payload to a Critic. Because the operational state was bound to the active context windows of the executing agents, this design resulted in high context bleed, high execution latency, and runaway token costs.

The New Architecture: A Decoupled Control Plane

The new architecture resolves these bottlenecks by decoupling the control plane. Hypatia was refactored into a State-Externalizing Harness, separating the execution state from the underlying agents.

The core is a centralized, external state ledger that acts as the master state machine. The agents were stripped of their operational history and converted into stateless functional transformers. Under this decoupled execution model, an agent wakes up, reads a specific target slice of the shared ledger, executes its localized semantic transformation, writes the result back to the central data plane, and immediately spins down.

Implementation in Hypatia

These three paradigms were synthesized to refactor Hypatia’s codebase, moving it from a flat sequential script to a decoupled control plane.

• RAPTOR (Hierarchical Trees for Global Understanding): Traditional RAG slices a document into disconnected pieces, losing the big picture. To fix this, a summary tree is constructed in parser.py. First, the PDF is sliced into 2,000-character blocks with a 300-character overlap so sentences and math formulas are not cut in half. Embeddings for all chunks are generated simultaneously using Gemini’s Native Batch Embedding API to avoid network rate limits. Then, a Consolidator step running a Map-Reduce loop deletes duplicate facts found across the chunks. To prevent this from acting as a lossy compression algorithm, a Hybrid Payload step extracts a Document Outline before chunking, and the internal schema explicitly forces the pipeline to preserve complex mathematical proofs and architectural edge-cases. This compiles a clean, condensed Level 2 (PaperFacts) summary that gives the agents global context without prompt bloat.
• MemGPT (On-Demand Context Paging): Instead of forcing the Critic agent to hold the entire 40,000-word paper in its memory just to check one specific fact, the raw document was stripped from its prompt entirely. The Critic is equipped with a tool in fact_checker.py that lets it “page in” specific paragraphs only when needed. Because standard vector search struggles to find exact strings like “v1.4.2” or a specific math variable, the tool uses a hybrid Reciprocal Rank Fusion (RRF) search engine:
  -It runs a rapid exact-keyword match.
  -It simultaneously runs a vector search.
  -It merges the results, grabbing the most relevant paragraph and injecting it into the Critic’s active context window.
  -To prevent retrieval failures when semantic queries miss, the Critic is also equipped with a retrieve_chunk tool, allowing it to explicitly page to specific chunk indices (e.g., fetching chunk_15 to read the rest of an adjacent paragraph).
• Harness-1 (Stateless Agents and External Memory): The burden of tracking operational history was removed from the agents. Instead of agents attempting to manage their own context logs, an external ledger is maintained in state.py called the HierarchicalMemoryMap. The agents are now stateless functional transformers. The system wakes an agent up, hands it a compressed snapshot of the ledger, and records the output directly to a state.json file on disk. To prevent data corruption when multiple agents execute concurrently via asyncio.gather, the system utilizes Pydantic’s model_copy(deep=True). This captures a frozen, atomic snapshot of the data in memory before initiating disk serialization, which ensures the underlying ledger is never corrupted by concurrent mutations.

The End Result

By externalizing state and managing memory actively, the refactored architecture delivers:

• Faster Execution: Utilizing asyncio.gather and batch embedding allows parallel extraction of chunks.
• Lower Costs: Agents process compressed snapshots instead of full papers, yielding token savings.
• Fewer Hallucinations: Deterministic state tracking and hybrid RRF search keep the agents grounded in the source text.
• Improved Debuggability: The entire memory tree and checklist history is serialized to state.json. Developers can trace a hallucination to its exact source node without reading massive prompt traces.

Trade-offs and System Limitations

However, moving from a flat prompt chain to a decoupled Map-Reduce harness introduces a new set of trade-offs:

• API Call Rate Limit: Fact extraction went from 1 massive API call to N+2 calls (one for each chunk, plus consolidation). This risks exhausting RPM rate limits.
  -Mitigation: A staggered asyncio.Semaphore delay was implemented to smooth out the parallel request fan-out.
• Lossy Compression Risks: Because the drafting agents rely entirely on the structured Level 2 summary node, they lack direct access to the raw source text. If the upstream analyst agent omits a subtle equation, or if the Map-Reduce consolidator flags an experimental variable as boilerplate noise, the final system output quality is bottlenecked by the accuracy of the early ingestion steps.
  -Partial Mitigation: The internal schema was hardened to force the preservation of complex math, and injected a pre-chunking Document Outline (Hybrid Payload) to prevent the loss of the global narrative.
• Retrieval Dependency: The Critic no longer holds the raw paper, relying on search queries. If a semantic query is poorly formulated, verification fails.
  -Mitigation: The Critic’s RRF search was augmented with a deterministic retrieve_chunk tool, allowing the agent to explicitly page to exact chunk indices when semantic queries miss.

References and Further Reading:

• Packer, C., Fang, V., Patil, S. G., Lin, K., Wooders, S., & Joseph, J. E. (2023). MemGPT: Towards LLMs as Operating Systems. arXiv preprint arXiv:2310.08560.
• Sarthi, P., Abdullah, S., Tuli, A., Khanna, S., Goldie, A., & Manning, C. D. (2024). RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval. arXiv preprint arXiv:2401.18059.
• Jiang, J., Pat, J., & Chroma Systems Team. (2026). Harness-1: Reinforcement Learning for Search Agents with State-Externalizing Harnesses. arXiv preprint arXiv:2606.02373.
• Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Hannaneh, H., & Manning, C. D. (2023). Lost in the Middle: How Language Models Use Long Contexts. Transactions of the Association for Computational Linguistics, 12, 148–163.
• Hypatia New Architecture — https://github.com/ashwini-anand/hypatia/tree/memory-map .