Enterprise AI Agent Memory: Remember or Forget?

AI agent memory is not one thing. Some of what a business wants an agent to “remember” belongs in a knowledge base, some is the record of what past runs did, some is context that should live only inside a single run — and some of it an agent should never keep at all.

Ask a team what they mean by “give the agent memory” and you get a list that quietly contradicts itself. They want it to remember the refund policy. They want it to remember what it did last Tuesday. They want it to remember the account it was just discussing three messages ago. They want it to remember a customer’s payment details — and, once you say that one out loud, they usually want to take it back.

That is four different requests wearing one word, and treating them as a single feature is how memory becomes a liability. A durable fact and a throwaway detail have opposite storage rules; the record of what an agent did is a different thing again; and the data you most want an agent to “keep in mind” is often exactly the data it should hold for the length of one task and no longer. The reason “AI agent memory” gets pitched as a magic capability is that it hides these distinctions. The reliable version makes them explicit.

This post is that version. It sorts what an agent should remember from what it should forget, maps each kind onto where it actually belongs, and treats the whole question as what it really is: a governance decision, not a feature you switch on.

What is AI agent memory?

AI agent memory is the set of mechanisms that let an agent carry information beyond the single message in front of it — the knowledge it can look up, the context it holds within a task, and the record of what it did on past runs. It is not a single store, and the useful distinction is not how much an agent remembers but which mechanism holds which fact and for how long.

The tempting picture is one big brain that accumulates everything the agent ever sees and gets smarter over time. That picture is wrong in a way that matters, because it implies durable facts and sensitive details and run outcomes all pool in the same place, retrievable forever. For a business, that is not a capability; it is an audit finding waiting to happen. What you actually want is the opposite of one big pool — a small number of clearly-scoped mechanisms, each with an obvious rule about what goes in it and what comes back out.

There are four kinds worth naming, and almost every argument about “agent memory” is really an argument about which one a given fact belongs in.

Durable company knowledge belongs in a knowledge base

Durable company knowledge is the stable, authoritative material an agent should answer from — policies, pricing, runbooks, contract terms, the meaning of internal words. The right home for it is not the model’s memory at all. It is a knowledge base the agent searches before it answers, so the response reflects your current document rather than whatever the model absorbed in training.

This is the single most important correction to the “give the agent memory” instinct. When someone says they want the agent to remember our refund policy, they do not want it baked into a recollection that drifts and cannot be cited. They want it grounded: retrieved fresh each time, current with whatever the document says today, and carrying the source so the answer can be checked. That is exactly what retrieval does and exactly what model recall does not — the difference between retrieval and training is the difference between knowledge you can update and correct and knowledge you cannot.

Two properties make grounding the reliable form of durable memory, and they are the properties every other kind of “memory” lacks:

It is current and correctable. Change the document, and the next answer changes with it. A fact that turns out wrong is an edit at the source, not a residue you have to hope the agent forgets.
It cites what it used. Each retrieved passage carries the document it came from, so “the agent remembered our policy” becomes “the agent read this policy, and here it is.” That is why AI answers need citations — a remembered claim you cannot trace is one you have to take on faith, and business facts do not get to run on faith.

So the first rule of agent memory is almost anticlimactic: the durable stuff does not go in “memory” in the folk sense at all. It goes in a source you maintain, and the agent looks it up.

Run history is the record of what an agent did

Run history is the persistent record of every unattended execution an agent has performed — when it ran, whether it worked, and what it produced. It is memory of a completely different kind: not what the agent knows, but what it did, and it is the mechanism that makes leaving an agent unattended a responsible thing to do.

When work moves onto a schedule with scheduled jobs, you stop watching it — that is the point. The month-end sweep and the Monday digest run whether or not you are present, which means “it ran and I trust it ran” now depends entirely on a record you can open afterward. In Insulin that record is the runs table on each job’s detail page: every trigger creates a run, newest first, and a row expands to that run’s result, its error, or the event payload that started it. Nothing an agent does unattended is supposed to vanish; if it ran, it is a row. (The mechanics of reading that table are covered in how to see past runs of a scheduled job.)

The reason to treat this as a distinct kind of memory is that it answers a distinct question. Grounding answers what should the agent know; run history answers what did the agent actually do, and can I reconstruct it. Conflate the two and you get the worst of both — an agent that “remembers” its past outputs as if they were facts, quietly treating last month’s result as this month’s truth. Keep them separate and each does one job: the knowledge base is the source, the runs table is the ledger.

Ephemeral task context should stay inside the task

Ephemeral task context is the working memory of a single conversation or run — the account you are discussing, the file you attached for this one question, the intermediate results the agent is holding while it finishes. It is real and necessary, and its defining property is that it should expire. It exists so the agent can follow a thread of work to the end, not so it can carry that thread into unrelated work later.

This is where the “one big brain” picture does the most damage. A detail that is useful inside a task — a customer’s account number while you resolve their ticket — becomes a hazard the moment it persists past the task and leaks into a different conversation, a different team’s agent, or a schema you now have to govern. Insulin’s agents let you attach files for one-off context precisely so this material can be scoped to the moment: it informs the current run and does not become durable. The discipline is to keep it that way — if a fact matters beyond one task, it is durable knowledge and belongs in a base; if it only matters for this task, it should not outlive it.

A note on session context, because it is easy to over-read. An Insulin agent can carry context between sessions, so a continuing conversation does not start cold — a convenience for the person working with it. That is context for continuity, not a store of record and not a place to park facts. The things that have to be right and checkable still sit in the knowledge base; the things that have to be reconstructable still sit in run history. Session context is the connective tissue, not the source of truth — and treating it as either of the other two is how a convenience turns into a claim you cannot back.

What an agent should not remember at all

Some data should never enter an agent’s durable memory in any form — regulated or personal data, secrets, payment details, anything under NDA whose exposure is a reportable event. The instinct to “let the agent remember the customer” runs straight into this, and the answer is not a better memory feature. It is a boundary drawn before anything is stored.

The safe default is to classify first and connect second: public and internal-non-personal material is what you want grounding an agent, while regulated or personal data needs your privacy team’s sign-off before it is wired to anything — and much of it should be reached transiently within a task, if at all, never persisted into a durable store. This is where memory stops being a technical question and becomes a governance one: the question is not “can the agent hold this,” it is “should anyone or anything downstream of this agent be able to see it later.” If the answer is no, it does not go into a base, and it does not become session context — it is used, if the task genuinely requires it, and then it is gone.

Here is the whole framework as a table — the four kinds of memory, where each belongs, and the risk of putting a fact in the wrong one.

Kind of “memory”	What it is	Where it belongs	Persistence	The risk of getting it wrong
Durable company knowledge	Stable policy, pricing, runbooks, contract terms	A knowledge base the agent searches before answering	Lives in a source you maintain; current and removable	Baked into model recall instead — stale, uncitable, hard to correct
Run history	The record of what each unattended run did	The runs table on the job’s detail page	Persistent ledger of executions	No record — an unattended agent nobody can reconstruct
Ephemeral task context	The account, file, or intermediate result for this task	Inside the single conversation or run	Expires with the task	Persisted past the task — leaks into unrelated work you now must govern
Sensitive / regulated data	PII, secrets, payment details, NDA material	Kept out; reached transiently only if the task truly needs it	Not persisted into any durable store	Stored “for convenience” — an exposure and a deletion problem you created

Read down the persistence column and the design becomes obvious: durable knowledge persists in a source you can edit, run history persists as a ledger you can read, task context expires, and sensitive data is never persisted at all. Four rules, one per kind — which is the entire discipline that “give the agent memory” was hiding.

Why memory is a governance decision, not a feature

The reason to resist “agent memory” as a single switch is that every hard question about it is a governance question in disguise: who can reach this later, can we correct it, can we delete it, can we prove what it did. A store that quietly accumulates everything answers all of those the wrong way — it makes reach broad, correction fuzzy, deletion uncertain, and proof impossible. The four-kind split answers them correctly by construction.

Governance is what decides it, so run the design through governance questions rather than capability ones:

Can we correct it? A wrong fact in a knowledge base is one edit at the source. A wrong fact fused into model recall stays until something retrains. This is why durable knowledge is grounded, not remembered.
Can we delete it? Remove a document from a base and it can no longer be retrieved — there is no residue in a weight matrix, because grounding reads at query time and trains on nothing. Data that was never persisted needs no deletion at all.
Can we prove what happened? Run history is the proof — a per-run record of result, error, and triggering payload. Human approval sits in front of the actions that matter: an Insulin agent proposes a plan and nothing runs until you approve it, so the consequential steps are a decision on the record, not something the agent did from memory.

None of this needs a special long-term memory product, and that is the point worth being blunt about. The reliable, auditable memory a business agent needs is assembled from parts that already have clear rules — a knowledge base for what it should know, a runs table for what it did, transient context for the task in hand, and a firm line around what it must never keep. Memory framed as a magic feature hides the decisions. Memory framed as governance makes you make them, which is the only version you can defend on day ninety.

Frequently asked questions

What is AI agent memory? AI agent memory is the set of mechanisms that carry information beyond a single message: knowledge the agent looks up, context it holds within a task, and the record of past runs. It is not one store, and each kind has its own rule.

What should an AI agent remember? Durable company knowledge — policies, pricing, runbooks — which belongs in a knowledge base it searches before answering, not in model recall. It should also keep a record of what each unattended run did, so the work can be reconstructed afterward.

What should an AI agent forget or never store? Regulated or personal data — PII, secrets, payment details, NDA material — should never enter a durable store, and task-specific details should expire with the task. If a fact only matters for one run, it should not outlive it.

Is a knowledge base the same as agent memory? A knowledge base is the reliable form of durable memory for facts. The agent searches it at answer time and cites what it used, so knowledge stays current and correctable — unlike facts baked into a model, which cannot be updated or traced.

Does Insulin have a long-term memory feature? Insulin does not treat memory as a single magic store. Durable knowledge lives in a knowledge base, unattended work is recorded in each job’s runs table, and an agent can carry context between sessions — each mechanism scoped to one job rather than pooled together.

Takeaways

“AI agent memory” is usually four requests in one word: durable knowledge, run history, ephemeral task context, and sensitive data. Each has an opposite storage rule.
Durable company knowledge belongs in a knowledge base the agent searches and cites — not in model recall, which is stale, uncitable, and hard to correct.
Run history is a different kind of memory: the ledger of what each unattended run did, which is what makes leaving an agent unattended defensible.
Task context should expire with the task, and sensitive or regulated data should never enter a durable store at all — classify first, and keep it out.
Memory is a governance decision, not a magic feature: design it around who can reach it, correct it, delete it, and prove what happened.

The short version: do not ask for one big memory. Ground the durable facts, keep the sensitive ones out, let run history be the record, and let task context expire. Explore Insulin knowledge bases, see how agents are scoped and approved, or get a demo.

Enterprise AI Agent Memory: What Should Agents Remember—and Forget?