First surfaced in Tandemly Briefing — 2026-07-14.

Agent Memory · Reinforcement Learning

Stop retrieving.
Navigate.

Researchers at Alibaba and ShanghaiTech organized user memory into four connected layers and exposed each layer as a callable tool. An RL-trained agent learns to move between those layers based on what it has found so far, stopping once it has enough evidence. Better answers, fewer unnecessary tool calls.

Core concept
Active memory navigation: treating long-term user history not as a store to query once, but as a structured space to explore in multiple steps, where each step is shaped by what the previous one found.
scroll to explore

A single query can't ask
what you don't know yet.

Most personalized agents still treat memory as a filing cabinet: submit a query, receive documents, proceed. That works until it doesn't.

Personalized conversational agents carry user history: preferences stated weeks ago, facts mentioned in passing, patterns that only emerge across sessions. The most common way to use this history is to embed the user's latest message, run a nearest-neighbor search, and inject whatever comes back into the prompt. It is fast, simple, and good enough for many queries.

The trouble is that the search is passive. The agent asks a question of its memory, takes what it gets, and moves on. There is no way for it to say "I found a preference record, but I want to verify it against the conversation where it came from" or "this topic track mentions something relevant; let me pull the detail." The model is a consumer of pre-selected evidence, not a navigator of a structured information space.

This matters because user memory is not a flat list of independent facts. It has natural structure. Raw conversation logs exist. Extracted factual records, typed by category, exist. Topic threads that connect related conversations across sessions exist. High-level user profiles that summarize patterns and preferences exist. Passive retrieval dumps a mix of these levels into context indiscriminately, and the agent has to sort out the signal on its own. That is work the retrieval layer could be doing.

The question this paper asks

What if the agent could choose which memory layer to inspect, and in what order, based on what it has found so far? Could that active navigation produce better answers than a single fixed retrieval pass?

A four-level pyramid
and an agent that climbs it.

NapMem introduces two ideas together: a structured memory pyramid and a trained policy for navigating it.

The first idea is the memory pyramid. NapMem organizes user history into four layers, linked by provenance pointers so that any claim at one level can be traced to the evidence at the level below it. The layers, from bottom to top, are: raw conversation transcripts, typed memory records (atomic facts organized by category), topic tracks (threads that connect related facts and conversations across time), and a user profile (a summary of stable preferences and behavioral patterns). The links between layers are bidirectional. The agent can start at the top and drill down, or start at the bottom and aggregate upward.

The second idea is exposing each layer as a callable tool rather than as a single retrieval endpoint. The agent can call "get user profile," then "get topic track for dietary preferences," then "get the specific conversation where the user mentioned a food allergy." Each call returns structured, typed output. The agent decides which call to make next based on what it has found so far. This turns memory access from a one-shot query into a multi-step decision process.

To teach the agent how to navigate this structure well, the researchers used reinforcement learning. The reward signal jointly optimizes two things: the quality of the final answer and the efficiency of the navigation path. An agent that reaches the correct answer in two tool calls is rewarded more than one that reached the same answer in eight. Over training, the model learns to stop exploring once it has gathered enough evidence, rather than calling every available tool by default.

4
User Profile
High-level summary of stable preferences, behavioral patterns, and long-term tendencies. The fastest entry point. Useful when the query is about a general preference rather than a specific event.
3
Topic Tracks
Threads that link related conversations and records across sessions. Useful when the query spans time or requires understanding how a preference evolved.
2
Typed Memory Records
Atomic facts extracted from conversations and organized by type (preferences, constraints, history, etc.). More precise than profiles; more compact than raw transcripts.
1
Raw Conversations
The original transcript, available when a specific exchange needs to be verified or when context around a fact matters for interpretation.

Fewer calls.
Better answers.

Across three memory-intensive benchmarks and a battery of non-memory tasks, NapMem's trained navigators consistently beat passive retrieval baselines.

Memory layers
4
Pyramid granularities
Benchmarks evaluated
3+
Memory and non-memory tasks
RL training effect
Fewer tool calls, higher accuracy

NapMem was evaluated on three memory-intensive benchmarks covering different angles of what it means to know a user: PersonaMem-v2 (multiple-choice questions about implicit preferences), LongMemEval (open-ended QA across five capabilities: information extraction, multi-session reasoning, temporal reasoning, knowledge updates, and abstention), and LoCoMo (open-ended QA on long-horizon conversational memory). The agent was also evaluated on standard non-memory reasoning and tool-use tasks to check whether the memory-specific training hurt general capability.

Across the memory benchmarks, NapMem was competitive with or better than passive retrieval baselines. The ablation results are the most informative part of the paper: all three components contributed independently. Active navigation alone (allowing multi-step tool calls without RL) improved over passive retrieval. Adding the memory pyramid (structured granularity) improved further. Adding RL training on top changed the navigation behavior: the trained model made fewer tool calls overall while reaching higher accuracy, indicating it learned to stop searching once sufficient evidence had accumulated.

On non-memory tasks, performance largely held. The RL training did not push the model toward over-relying on memory tools in contexts where they are not useful. That is a meaningful finding: it means the navigation policy learned by the model is selective, not compulsive.

Honest limitation

The paper does not surface absolute accuracy numbers prominently in publicly available materials. The relative findings (active navigation beats passive, granularity contributes, RL improves selectivity) are clearly supported by the ablation structure, but direct comparison to specific external baselines requires reading the full paper at arxiv.org/abs/2607.05794.

What this means
for builders.

The core lesson is architectural: if memory has structure, expose that structure as callable tools rather than collapsing it into a single retrieval endpoint.

1
Audit your memory layer for exposed granularities
Most production memory systems expose one endpoint: embed the query, return top-k documents. Ask how many distinct granularity levels your system could expose as separate tools. If the answer is one, you are forcing the model to be a passive consumer. Consider splitting into at least two levels (summary and detail) and exposing each as a distinct callable.
2
Use typed structured output on memory tool calls
When a memory layer returns records, make those records typed. A "dietary constraints" record is more useful than a chunk of text with dietary information buried inside it. The NapMem pyramid's typed memory records work precisely because the agent can route to a type category directly, rather than parsing unstructured context to find the relevant field.
3
Include navigation efficiency in your RL reward
If you are RL-training an agent that uses memory tools, add a navigation efficiency signal to the reward alongside final-answer quality. A joint reward teaches the agent to stop searching once it has enough evidence. Without it, a trained agent will use more tool calls than necessary, because "calling more tools" looks safe from the perspective of a reward that only measures final accuracy.
4
Evaluate on multi-dimension memory benchmarks, not just recall
PersonaMem-v2 and LongMemEval probe meaningfully different things. PersonaMem tests whether the agent understands implicit preferences; LongMemEval tests five distinct capabilities including temporal reasoning and appropriate abstention. Evaluating only on recall-style tasks will not surface whether the agent correctly declines to use stale or irrelevant memory. Use both families of benchmarks before concluding a memory system is production-ready.
5
Verify non-memory capability preservation after RL training
Any RL training on memory-specific behavior risks narrowing the model's behavior in contexts where memory is not the right tool. The NapMem results show preservation on non-memory tasks, but this should be verified explicitly on your own distribution. Include a non-memory evaluation suite in your RL training loop from the start.

Read and replicate.

1
Read the paper
Xu, Y., Sun, Y., Liu, Y., Zhou, M., Qiao, J., Ma, L., Tang, K., Wang, W., Jiang, X., & Jiang, G. (2026). From Passive Retrieval to Active Memory Navigation: Learning to Use Memory as a Structured Action Space. Alibaba & ShanghaiTech University. arXiv:2607.05794. The full paper includes absolute benchmark numbers and full ablation tables not available in the abstract alone.
2
Map your current memory system to the four NapMem layers
Take your existing user memory store and ask: what corresponds to raw conversations, typed records, topic threads, and a user profile? Even if your system only has one or two of these, naming them explicitly makes it easier to reason about what is being collapsed into a single retrieval call and what you could expose separately.
3
Run LongMemEval on your personalized agent
LongMemEval covers five capability dimensions (information extraction, multi-session reasoning, temporal reasoning, knowledge updates, abstention) and is specifically designed for chat-assistant memory. It reveals failure modes that single-session recall benchmarks will miss, particularly around update tracking and appropriate non-response when the relevant memory no longer holds.
4
Prototype a two-tool memory interface
Before building a full pyramid, test the core idea with two tool calls: one that returns a high-level summary (profile or topic track) and one that returns supporting detail (typed records or transcript excerpts). Measure whether the agent uses both tools selectively or defaults to one. If it defaults to the summary and ignores the detail tool, you have a calibration signal for your reward design.
5
Add PersonaMem-v2 for implicit preference evaluation
PersonaMem-v2 tests whether the agent infers implicit user preferences, not just explicit stated facts. This is the harder and more realistic test for personalization: users do not always state what they want directly. Running both PersonaMem-v2 and LongMemEval gives you a more complete picture of memory capability than either benchmark alone.