The gap between a demo that impresses a room and an agent your business quietly depends on is a checklist. Six things have to be true — grounding, permissions, approvals, monitoring, rollout, and ownership — before an AI agent stops being a pilot and starts being production.
Most AI agents die in the space between “that was a great demo” and “we run our month-end close on it.” The demo was easy: one person, one happy path, no consequences if it was wrong. Production is the opposite — many people, real data, and a downstream number that has to be right. A ninety-day adoption plan tells you the strategy for crossing that gap. It does not tell you the single moment when the agent is allowed across.
This post is that gate. It is a go-live checklist you can run as one cross-functional signoff — product, IT, security, and operations reading the same six lines — with each line grounded in something you can actually configure and inspect in Insulin, not a governance abstraction.
What has to be true before an AI agent goes to production?
Six things have to be true: the agent is grounded in the right knowledge, scoped to only the systems it needs, gated by approvals on anything consequential, monitored through a durable run history, rolled out evaluate-first rather than all at once, and given a named owner. Miss any one and you have a demo with a wider blast radius, not a production system.
The rest of this post is one section per item, in the order you should verify them. Treat each as a line on a signoff sheet: it is either demonstrably true or the agent is not ready.
1. Grounding: is the agent answering from your material, not its guess?
An agent is production-ready on grounding when its answers come from your documents with cited sources you can check — not from the model’s general knowledge. In Insulin you attach knowledge bases to an agent so it answers “from your pricing, contracts, and runbooks,” and every answer carries the sources it drew from.
The check is not “does it sound right.” A fluent wrong answer is the most dangerous output an agent can produce, because it is the one nobody double-checks. The check is: open a hard question, then open the citation. If the cited passage supports the answer, the grounding holds. If the agent is confident and the citation is thin or absent, the knowledge base is wrong, and that is fixable before go-live rather than after a customer sees it.
One thing this line is not: Insulin grounds at retrieval time and does not train on your data. So “grounding” here means the agent reads your current documents when it answers — update the knowledge base and the next answer reflects it, with no retraining step to wait on.
2. Permissions and scope: can the agent only reach what its job needs?
An agent is production-ready on scope when it can reach exactly the integrations its job requires and nothing more. When you create an agent in Insulin you allowlist the integrations it may use — scope is “the boundary, so an agent cannot reach data outside its job.” A deal-desk reviewer gets Listings, Offers, and Salesforce; it does not get the finance systems it has no business reading.
Scope is the security line, and it is easiest to get wrong by being generous. The instinct under deadline is to grant broad access “so it doesn’t get blocked.” Resist it. The right question at the gate is not “what might this agent ever need” but “what does this job need,” and the answer is almost always narrower than the demo had.
This matters doubly because scope is inherited. A scheduled or event-driven job keeps the integration scope you granted the agent — the job “inherit[s] everything from the agent you pick, which is what makes them safe to leave running.” So the boundary you set here is the boundary that holds when the agent is running unattended at 3 a.m. Get it right once, up front.
3. Approvals: does a human confirm anything consequential?
An agent is production-ready on approvals when nothing consequential happens without a human confirming it. Insulin’s model is “an agent proposes; you approve” — tool execution runs through an approval workflow, and “nothing runs until you approve.” You approve or reject a plan before any tool fires.
The discipline is deciding which actions are consequential and gating those. Reading a record and drafting a summary can run freely. Sending an email on someone’s behalf, writing to a system of record, or moving money should not. Insulin already puts agent-initiated Slack messages behind an approval gate for exactly this reason — a message sent in your name is an action, not a draft, and it should wait for a human. The August 3, 2026 release in the Insulin changelog shipped that gate.
The go-live check: list the agent’s consequential actions and confirm each one either requires approval or is deliberately, explicitly trusted to run on its own. A blank “everything auto-runs” is not an answer — it is the absence of one.
4. Monitoring and run history: can you reconstruct what it did?
An agent is production-ready on monitoring when every execution leaves a record you can open after the fact. The moment you put an agent on a schedule you give up watching it, so trust has to come from a durable log instead. Every Insulin job has a detail page with a runs table — one row per execution, showing when it started, its status, and its duration — and expanding a row reveals that run’s result, its error if it failed, or the event payload that triggered it.
This is what makes an unattended agent auditable rather than a black box. Nothing runs invisibly: even a filtered or rate-limited trigger is a row, so the absence of a run is as visible as its presence. A failed run is not a silent gap; it is a row you can open to the exact error. We covered the mechanics in how to read a job’s run history; the unified job detail page and runs table shipped in the July 27, 2026 changelog entry.
The check: before go-live, confirm you can answer “what did it do last Tuesday, and did it work?” without hunting. If the answer is a row you can open, the monitoring line holds.
5. Rollout: are you shipping evaluate-first, org-wide, and idempotently?
An agent is production-ready on rollout when you have proven it on a single account first and can then push the same component org-wide without re-authoring it. The wrong way to standardise is to pick an agent in a meeting and hand it to two hundred people who have never used it. The right way is boring: one person runs it on real work, and it goes wide only after it earns that trust.
Insulin’s agent marketplace is built around that sequence and gives you three properties the gate should confirm:
- Evaluate-first. Install the agent for yourself; it appears in your workspace only, so you can test it against real work with a blast radius of one before anyone depends on it.
- Org-wide as one action. Rolling out installs a workspace-wide copy of the same component for everyone, so every team starts from the specialist you proved rather than rebuilding their own.
- Idempotent. Re-installing the same item does not create a duplicate, so an org-wide roll-out on top of the account you evaluated on leaves you with one copy, not two to reconcile.
The evaluate-first org-wide rollout is the whole method; org-level agents and skills shipped in the June 29, 2026 changelog entry. The check: you can name the account it was proven on, and confirm the org-wide install ships that exact component.
6. Ownership: is there a named person accountable for it?
An agent is production-ready on ownership when a specific person — not a team, not “IT” — is accountable for its instructions, its knowledge, and its runs. A production agent is a system someone maintains, and role-based access is how you make that concrete: Insulin agents carry roles (admin, editor, user), so you can say precisely who may change the agent’s configuration and who may only use it.
Ownership is the line teams skip because it feels like paperwork, and it is the one that fails quietly. An agent with no owner drifts: its knowledge base goes stale, a run starts failing, and nobody is on the hook to notice. Name the owner at the gate and the whole checklist gets a home — the person who checks the grounding, holds the approval decisions, and reads the run history when something looks off.
The check is a single field: who owns this agent? If the honest answer is “nobody yet,” it is not ready, no matter how good the demo was.
The go-live checklist
One signoff line per item. Every line is either demonstrably true or the agent stays in pilot.
| # | Line | Ready when… | Configured in Insulin via |
|---|---|---|---|
| 1 | Grounding | Answers cite your documents; a hard question checks out against its citation | Knowledge bases attached, retrieval-time |
| 2 | Permissions / scope | The agent can reach only the integrations its job needs | Integration allowlist at agent creation |
| 3 | Approvals | Every consequential action requires a human confirm or is explicitly trusted | ”Propose, then approve” tool workflow |
| 4 | Monitoring | You can open any past run and see when, whether it worked, and what it did | Per-job runs table (Started / Status / Duration) |
| 5 | Rollout | Proven on one account, then the same component pushed org-wide, idempotently | Marketplace: install for self, then org-wide |
| 6 | Ownership | A named person is accountable for config, knowledge, and runs | Role-based access (admin / editor / user) |
Frequently asked questions
What has to be true before an AI agent goes to production? Six things: the agent is grounded in the right knowledge, scoped to only the systems it needs, gated by approvals on consequential actions, monitored through a durable run history, rolled out evaluate-first, and given a named owner. Miss one and it is a wider-blast-radius demo, not production.
How do you keep a production AI agent from accessing the wrong systems? Allowlist only the integrations its job needs when you create the agent. Scope is the boundary that stops it reaching data outside its job, and a scheduled or event-driven job inherits that same scope, so the boundary holds when it runs unattended.
How do you make sure an AI agent asks before doing something consequential? Put consequential actions behind approvals. In Insulin an agent proposes and you approve; nothing runs until you confirm. Reading and drafting can run freely, but sending a message, writing to a system of record, or moving money should wait for a human.
How do you monitor an AI agent that runs unattended? Read its run history. Every Insulin job has a runs table with one row per execution showing when it started, its status, and its duration; expand a row for the result, error, or triggering payload. Even filtered or rate-limited runs are rows, so nothing disappears silently.
What is the safest way to roll an AI agent out to a whole organization? Evaluate-first. Install it on one account, prove it on real work, then roll the same component out org-wide. Installing is idempotent, so the org-wide rollout on top of the account you tested leaves one copy, not two to reconcile.
Who should own a production AI agent? A specific named person, not a team. Role-based access (admin, editor, user) makes it concrete: one accountable owner maintains the instructions and knowledge, holds the approval decisions, and reads the run history when something looks off. An unowned agent drifts quietly.
Takeaways
- Production readiness for an AI agent is a six-line gate: grounding, permissions, approvals, monitoring, rollout, and ownership. Each line is demonstrably true or the agent stays in pilot.
- Grounding is checked against citations, not vibes; scope is set by allowlisting only the integrations the job needs, and the job inherits that scope when it runs unattended.
- Approvals gate the consequential actions — sending, writing, paying — while reading and drafting run freely; “an agent proposes, you approve.”
- Monitoring means a durable runs table you can open per execution, so an unattended agent is auditable rather than a black box.
- Roll out evaluate-first and idempotently, and give every production agent one named owner via role-based access — an unowned agent drifts quietly.
Run this checklist as a single cross-functional signoff and “should we ship it” stops being a debate and becomes six answerable questions. When you are ready, scope and configure the agent, prove it with evaluate-first org-wide rollout, and watch it in production through its run history.
Stay Updated
Get the latest Cloud GTM insights, product updates, and marketplace strategies delivered to your inbox.