Large language model agents have improved at handling complex tasks but an important feature keeps lagging: memory. These agents treat each input in isolation. They cannot carry forward details about user preferences, past interactions, or project status. This stateless approach limits personalization and makes them less effective in roles that rely on consistent context. In real-world scenarios such as project management or long-term tutoring, agents that forget prior conversations must repeatedly rebuild background, slowing down workflows.
MIRIX AI has rolled out MIRIX, a memory management system that equips LLM-based assistants with lasting memory across multiple modes. This framework moves beyond simple text logs by integrating six distinct memory domains, each managed by its own Memory Manager. Developers can tweak storage policies without redeploying models, and the same agents can run on-device with limited resources or scale up in the cloud.
Rather than storing only raw text, this design splits memory into six domains. Each domain runs under a dedicated Memory Manager:
- Core Memory: stores data about agent and user. It splits into persona and human info. Persona holds the agent’s profile, tone, and usual behavior. Human covers user-specific fields such as name, likes, dislikes, and social connections. It also caches agent state such as active tasks or conversation phase.
- Episodic Memory: records events and interactions with timestamps. Each record includes fields like event_type, summary, detailed notes, participants, and a timestamp. Records can be filtered by date range or event_type.
- Semantic Memory: contains conceptual knowledge, entity relations, and knowledge graphs. Entries are tagged by type, provided with a summary, expanded details, and a source reference. Graph queries let it fetch related entities in a knowledge graph.
- Procedural Memory: holds structured task guides and workflows. It encodes step-by-step instructions often in JSON format for easy editing and execution. Workflows can trigger external services after completing a step.
- Resource Memory: maintains links to outside materials—documents, images, audio—with metadata such as title, description, resource type, and the content or link itself. Thumbnails for images and previews for documents help with quick browsing.
- Knowledge Vault: protects sensitive records like passwords, API keys, or private contacts. Data in this vault carries strict permission labels and is accessed under tight controls. Entries can be revoked or redacted to comply with retention policies.
Above these six managers, a Meta Memory Manager directs traffic. It decides where to send each data request, stacks entries in a hierarchy, and handles memory retrieval from the appropriate store. Other agents for chat handling and user interface communicate with this coordinator to fetch or update memory.
A security layer tags entries by sensitivity. All data in the Knowledge Vault stays encrypted at rest and in transit. Audit logs record each memory read or write, supporting compliance with regulatory standards.
A standout mechanism in MIRIX is Active Retrieval. When a user sends a prompt, the system identifies topics automatically, pulls relevant snippets from all memory domains, and marks them for inclusion in the next system prompt. This cuts down on reliance on the model’s built-in knowledge and anchors responses in the most recent user data. That injection mechanism supports prompt templates that adapt to each memory type. Retrieval can use any of several modes—embedding_match, bm25_match, string_match—with room to plug in new methods over time.
MIRIX is delivered as a cross-platform assistant built with React-Electron for its graphical interface and Uvicorn powering the API backend. It watches screen activity by capturing a screenshot every 1.5 seconds, discarding duplicates. Memory updates kick in when 20 unique images accumulate, which happens roughly each minute. Screenshots stream to the Gemini API for visual processing, enabling the system to ingest visual context within five seconds. The backend exposes endpoints for manual memory queries and supports rate limiting to prevent overload.
Users see a chat window that weaves in data from all memory stores to craft replies that feel personal and coherent. Lists or tree views present semantic and procedural content so users may review or edit what the agent has saved about them.
The system has been tested on two benchmarks. ScreenshotVQA is a visual Q&A test set that stresses memory across a series of high-resolution screen images. MIRIX delivered a 35 percent gain in LLM-as-a-Judge accuracy over retrieval-augmented baselines like SigLIP and Gemini, and cut storage needs by 99.9 percent compared to text-heavy approaches. LOCOMO evaluates how well an agent holds conversation history. MIRIX hit 85.38 percent average accuracy, outpacing LangMem and Mem0 by more than eight points and brushing up against the full-context ceiling.
Thanks to its modular architecture, the system scales across text-only and multimodal tasks, runs partly on-device or in the cloud, and can extend to wearable gadgets like smart glasses. Potential scenarios include real-time meeting recaps, detailed location logs, and dynamic habit-tracking.
A novel element is the Memory Marketplace, a peer-to-peer network for sharing or trading memory segments in a secure, encrypted environment. Users keep control over data through granular privacy settings, end-to-end encryption, and decentralized storage.
- What makes MIRIX different from existing memory systems like Mem0 or Zep?
- It splits memory into multiple domains beyond raw text, supports visual and other modalities, and uses a multi-agent retrieval layer for fast, accurate access across large datasets.
- How does MIRIX achieve low-latency updates for visual inputs?
- Streaming uploads to the Gemini API let it refresh screenshot-based memory in under five seconds during live sessions.
- Can MIRIX work with closed-source LLMs like GPT-4?
- It operates as an external service rather than a plugin, making it compatible with any LLM, including proprietary models such as GPT-4 and Gemini.