AI agent memory: How to build AI agents with persistent context

If you had to walk a new colleague to the printer every morning for six months, you’d start to worry. If they could recite every previous walk in perfect detail but still needed escorting, their recall is excellent…but you’d still be getting up from your desk.

AI agent memory, at present, has a version of this forever printer-walk. Letta’s CEO Charles Packer says in The New Stack that memory benchmarks are saturated because they measure recall, which he considers solved. The open problem is an agent improving at its work through the context it keeps, with no retraining involved.

His co-founder Sarah Wooders compares it to starting a job—benchmarks hand the agent all the context it needs—task after unrelated task, which is nothing like how people work. A new hire might need walking to the printer. But three years in, you’d hope to hand them a one-line brief without getting up from your desk.

That’s the standard to build toward, and the aim behind Brain, our self-improving memory system for Perplexity Computer, which organizes past work and corrections into an evolving knowledge wiki.

We’ll cover what an agent should remember and how to test whether it helps the agent do better work.

What is AI agent memory?

AI agent memory is the information an agent retains from previous interactions and work, along with the mechanisms for updating and using it. The persistent memory context carries that information across sessions, so closing a conversation doesn’t mean starting the next task from scratch.

Most of what ships under that name remembers the user. Your preferences, your contacts, your role, the fact that you want PDFs rather than PNGs. That’s the easy half.

When we built Brain, the memory system behind Perplexity Computer, we focused on the work itself. Which approach succeeded? Which source wasted twenty minutes? What did you have to correct after the agent announced it was finished? That history gives the next task a better starting point.

Brain stores this information as files in the agent’s sandbox, organized into three layers:

• sessions/ holds raw transcripts and their summaries.
• notes/ holds distilled information organized by topic.
• knowledge/ holds the wiki itself; linked Markdown pages about people, projects, and concepts.

None of that improves on its own. Overnight, background agents we call “Dream” read the day’s sessions, summarize them, and revise the wiki wherever the picture has changed. We stage and check those edits before committing them through Git, so there’s a diff to inspect when something looks off.

Does it make the agent better at the job? In our 30-day paired evaluation, Brain-enabled sessions improved correctness by 9.3 percentage points and recall by 8.9, while using roughly 15% fewer tokens.

Expand

Source: X

What are the types of agentic memory?

There are two broad kinds, and the split is much like the one you’d use for a person: what you’re holding in your head right now, and what you’ll still know next month.

Short-term memory

Short-term memory holds the context for the current conversation or task. Your request, recent messages, the result of a tool call, the document the agent just opened. Enough information to work out what to do next.

You’ll also see this called working memory. For an LLM, the context window holds the information available during a particular model call. The application may maintain an additional task state and decide which parts to supply at each step.

And short-term doesn’t necessarily mean disposable. A framework can save the conversation’s state and resume it tomorrow—it still belongs to that thread. But whether another task can use the information is a separate design decision.

Long-term memory

Long-term memory makes information available across sessions. The CoALA paper borrowed three categories from cognitive science, giving agent builders a vocabulary for experiences and know-how:

• Episodic memory is what happened: This includes past sessions, the steps an agent took, the result it got. In Brain, sessions/ provides this history through raw transcripts and their summaries.
• Semantic memory is what the agent takes as true: This could be facts about the user, their projects, and their world, distilled from the conversations. Brain’s knowledge/ wiki holds this kind of information, with links back to the evidence.
• Procedural memory is how to do things: These are prompts, reusable workflows, the Skills Perplexity builds to give agents the instructions and tools a job requires. You may already have some of this without calling it memory.

The interesting work happens between the layers. A session records that you rejected a PNG. Your instruction to use PDFs becomes a preference the agent can carry forward. A correction to the export process becomes something it can follow next time.

That conversion, from “this happened” to “this is true” to “this is how we do it,” is part of the learning problem we opened with, and it’s where Brain’s overnight Dream agent runs spend their effort.

Say someone microwaves fish in a shared kitchen. There are complaints, a window opens. Eventually, a sign appears: NO FISH.

Six months later, the offender has moved out, but the sign hasn’t. That’s persistence. You can explain the memory types through what survives:

• Working memory is what you’re keeping track of while making lunch, like the soup needs stirring.
• Episodic memory is the fish incident; the smell that followed everyone into the hallway.
• Semantic memory is the knowledge you took from it—reheating fish can leave a lingering smell in a shared kitchen.
• Procedural memory is the instruction governing what happens next. Do not microwave fish here. The sign lets someone who missed the entire incident follow the rule.

There’s also one more distinction, because it changes what we’re building. A survey first published in December 2025, Memory in the Age of AI Agents, says that short versus long isn’t enough—it also sorts memory by form:

• Token-level: This is the explicit content the agent reads. Files, stored records, retrieved chunks.
• Parametric: This is information encoded in learned weights. Updating it requires training or model-editing methods.
• Latent: This is information retained in internal representations without being written out as readable text.

We're focusing on token-level memory here; that is, information stored outside the model and read into its context when needed. Brain’s Markdown wiki is one example.

Expand

When we update a page, we change what the agent can read on its next task, but its underlying weights stay the same. That’s Packer’s point from the introduction: getting better through the context the agent keeps, without retraining.

How do the agent memory types work together?

An agent can revisit an earlier attempt, see where it went wrong, and load the instructions you gave it afterward. There are several kinds of memory involved in the same task, and they don’t each need a turn or a separate database.

Here, persistence determines what remains available after the conversation ends, while retrieval determines whether the agent ever sees it again.

In Brain, we preserve that route from the lesson back to the evidence. Say you ask Computer to continue a project from last week:

• Brain’s index tells it which memory pages exist and where to find them.
• Brain opens the project page to read the saved decisions and instructions.
• If a note says you rejected an approach, the agent can follow its citation back to the conversation and check why.

The links to related pages let it look up the people or other projects involved. The agent does this by reading and searching files. grep, for example, lets it find mentions of a project by name.

We’ve also tested what agents gain from being able to explore files. In an internal benchmark of 150 financial-document questions, an exploration agent using standard file tools reached an answer score of 0.627, compared with about 0.37 for semantic search over the same corpus.

Expand

Source: X

How to improve AI agent memory?

A May 2026 paper by Orogat and Mansour identifies four recurring failures in agent memory: the store keeps growing, outdated beliefs go unrevised, information gets forgotten when space runs out, and retrieval reads without learning anything from the visit.

Their proposed fix is to stop treating agentic memory as storage and treat it as a “state” that evolves through four operations: ingest, revise, forget, retrieve. That’s our checklist for this section.

Be selective about what gets remembered

Once a “bad” memory’s in, every step inherits it.

Workday AI Research tested a selective approach on roughly 1,500 candidate memories from the LoCoMo benchmark, asking five questions before keeping any of them:

• Is it useful? Will it plausibly come up again?
• Is it trustworthy? Did the conversation actually say this, or did the model fill it in?
• Is it new, or already stored?
• Is it fresh?
• Is it a durable fact or a passing state?

Only the first question needed a model call. The other four ran as rules in under 65 milliseconds combined. The method achieved 12% higher precision on keep-or-drop decisions while retaining 97% of relevant memories.

The biggest contributor was the last question—remove the distinction between durable and temporary information, and decision quality dropped roughly 18%.

Revise the answer without losing the history

Record when a fact became valid, which source supports it, and what it replaced. If the sources disagree, keep that disagreement visible until there’s enough evidence to resolve it. The newest document can also be wrong.

Brain updates the wiki using source documents, connector results, and user corrections. Each memory entry links back to its source. We need the agent to find the current answer and have somewhere to look if that answer seems suspicious.

Forget on purpose

The paper calls one failure mode “capacity-driven forgetting”: information gets removed to satisfy a storage limit, rather than because it’s stopped being useful.

Back to the shared kitchen. After six months, nobody cares who microwaved the fish or how long the window stayed open. But there's still a job for the sign.

If the next person makes lunch, the rule will be available. You just archive the complaints.

Workday’s freshness and information-type checks start before storage, filtering what deserves to become a memory. In Brain, we also revisit existing knowledge as source documents change and users make corrections. In either case, the content makes it necessary to intervene.

Make retrieval feed the next update

The agent retrieves a fact, uses it, and leaves the store exactly as it found it. Now, unless the system captures what happened next, nothing records whether that fact helped or sent the agent down another dead end.

Record the outcome, the source, and any correction. When we built Brain, we made that work history part of what it remembers: which approach succeeded, which source led nowhere, and what the user corrected after the agent announced it was finished. Brain incorporates that experience into its periodic wiki updates, so later tasks can use it.

None of this, however, shows up on a recall benchmark. The way to know whether the loop is working is to give the agent a task it can only complete with memory and count how often it completes it, with cost and latency attached. That’s what DolphinBench does, and it’s why the top scores are still in the seventies.

How to build an agent with persistent memory using Perplexity

Perplexity’s Agent API doesn’t ship a self-maintaining memory system like Brain. We’d rather say that upfront. But what it does give you is the pieces of the memory you need: a place to keep procedures, a way to carry context between turns, and tools to read from and write to a store you control.

Start with procedural memory

A custom Skill packages instructions, procedures, and supporting files under a version number. A platform team can write its deployment checks, rollback criteria, and reporting format once, then have applications reference the same Skill. When the procedure changes, the team uploads a new version.

Expand

Source: X

A Profile goes one level up. Profiles save the agent’s model, instructions, tools, Skills, connectors, and run settings under a single versioned ID.

We added both in September, along with managed connectors, so a project administrator can configure an approved GitHub or Slack connection once and let applications reference it instead of storing separate credentials. When access changes, the administrator updates the connection in one place.

Why does this count as memory? Once again, back to the microwaved-fish kitchen. Whoever arrives next can follow NO FISH without having witnessed the complaints.

A Skill does the same job for an agent, with room for the steps, exceptions, and checks that won’t fit on a microwave door. Skills are written down, versioned, and available the next time the task comes up.

We use Skills to guide Dream’s memory updates, too. But what a Skill doesn’t do is learn on its own. The instructions change when a new version is uploaded, whether by a person or an automated process.

If you want the agent to notice that the rollback check keeps failing on one service and adjust, that’s the sessions-to-wiki loop from earlier.

You need to retain what happened, work out what should change, and make the correction available next time. That needs the next two pieces.

Carry working memory between turns

The agent runs a deployment check. You ask, ‘Why did that fail?’ A reasonable follow-up, provided the agent can still see what “that” refers to.

Our Agent API supports two ways to carry conversation context forward:

• Replay the conversation yourself: Include earlier messages in the next request’s input array. You decide what to keep, summarize, redact, or drop before sending it.
• Continue from a completed response: Save its ID and pass it as previous_response_id with the next message. We keep the conversation state server-side, so your application doesn’t have to resend the earlier turns. You can continue from the latest response in the chain.

Replay gives you control over what the agent sees, and continuing by ID saves you the work of assembling the conversation again. Either way, supply any new instructions, tools, or model settings needed for the next response.

Bring your own episodic and semantic store

This is the part the Agent API leaves to you.

One route is function calling. You define the tools, the agent requests a call, and your application executes it and returns the result. For memory, a simple starting point is a read operation before the agent acts and a write operation afterward.

Log only the fact that the agent retrieved, and you still don’t know whether it helped. Log the outcome, the source, and whatever the user corrected, and you have the start of an episodic layer.

Our MCP support gives you another route. If a compatible remote MCP server exposes the operations you need, the agent can call them without you wrapping each one as a custom function.

Our connectors give the agent access to evidence behind the memories. GitHub, Slack, Google Drive, Datadog, Linear, and Notion are in our managed catalogue, and you can add custom connectors for internal systems.

Finally, decide when the history gets reviewed, and the current account gets revised.

Test what survives a fresh session

Now close the conversation. Start a new request without replaying the old messages or passing previous_response_id, but give the agent access to the same memory store. Ask it to do related work without repeating the correction.

If it needs the reminder, you haven’t yet finished.

For the deployment agent, build a few tests around what should carry forward:

• The correction survives: It retrieves and uses the verified rollback command for the affected service.
• The correction stays in scope: It leaves other services’ procedures alone.
• New information replaces old guidance: When the command changes again, it uses the current version and can trace the change to its source.

Our regression-testing cookbook shows how to save agent outputs and retrieved evidence, check them against expected behavior, and compare runs after changing instructions or models.

Keep the model, Profile, and Skill versions fixed while comparing runs with and without memory. Start each comparison from a known snapshot of the store; otherwise, one test may teach the agent the answer to the next.

When something fails, save the case. The next change should have to pass it.

Or skip the build: Brain in Computer

If what you want is the finished loop rather than the parts, that’s Computer. Brain runs underneath it, learning from four sources: your sessions, connected tools, the files and artifacts you make, and your corrections. Brain’s background updates reinforce what still holds, revise what’s changed, and mark stale information.

You don’t have to build that loop yourself. You just do the work.

Memory keeps preferences and details you’ve shared. Brain connects projects, people, decisions, and unfinished work to the evidence behind them.

You can inspect what it learned in the Memory view, follow entries back to their sources, and edit or delete them. Brain uses your own activity, excludes incognito sessions, and has its own toggle.

Expand

At Rho, for example, an invoicing project’s decisions were scattered across Slack, Notion, Jira, Figma, and Google Docs. The team set up Computer to review changes overnight, flag inconsistencies, and publish a morning digest in Slack.

Their software engineer Oktawian Jurkiewicz noticed that Computer was also building memory for the project. “We didn’t define it,” he says. “Computer figured out which data points it should store from each source.”

The team reports cutting weekly meeting time from 2.5 hours to 15 minutes and estimates saving 120 work hours over the 12-week project.

The 120 hours are credited to the whole workflow: connectors, overnight runs, and the Slack digest. How much of the time saved came from memory alone? We can’t yet quantify that.

But what we can advise is that for your own build, return to the test from earlier—run equivalent tasks with the memory layer on and off, then compare completion rates, cost, and time, including the time someone spends correcting the result.

AI agent memory FAQs

Can AI agents have memory?

Yes. AI agents can retain information across sessions through stored conversations, facts, preferences, and reusable procedures. The system must retrieve that information when it’s relevant and update it when circumstances change.

Brain in Perplexity Computer, for example, builds working context from sessions, connected tools, files, and user corrections.

How much RAM does an AI agent use?

There’s no single requirement. If your agent calls a hosted model through the Perplexity Agent API, you don’t need to load that model into your own RAM. Your application and local tools still need memory.

A local model run is different. At four bits per parameter, eight billion parameters amount to roughly 4 GB of raw weight data. The actual memory needed is higher once you include quantization overhead, context processing, and the software running the model.

What is the best memory for an AI agent?

The best setup depends on what the agent needs to carry forward. A conversation state supports follow-ups; episodic records preserve what happened; semantic memory holds current knowledge; procedural memory supplies reusable instructions.

Most ongoing work benefits from a combination. We use a linked, source-backed wiki for Brain. For your own build, test whether the setup helps the agent complete tasks, apply corrections, and retire outdated guidance.