Agent releases need trajectory gates

Shipping an agent is not the same as shipping a chat response. Once the system can call tools, read records, update tickets, trigger workflows, or compose actions across multiple steps, the release question changes. It is no longer only, “Did the final answer look right?” It becomes, “Did the agent take an acceptable path to get there?”

That distinction matters because a plausible answer can hide a dangerous trajectory. An agent can summarize a customer account correctly after calling the wrong system first. It can produce a clean recommendation while skipping the evidence fetch. It can succeed in a demo by overusing tools that will be rate-limited in production. It can answer safely in one turn, then drift during a multi-turn exchange.

The thesis for product teams is simple: agent releases need trajectory gates. Not gates that freeze every reasoning step, and not brittle tests that punish useful flexibility. A trajectory gate defines the expected operational path, the tolerated variance around that path, and the release thresholds for answer quality, groundedness, safety, tool use, and regression.

That is also how the evaluation vocabulary is maturing. Google’s Agent Development Kit documents criteria for tool trajectory, response matching, groundedness through hallucination checks, safety, multi-turn task success, and tool-use quality. Its tool trajectory criterion explicitly supports exact, in-order, and any-order matching, which is the practical language teams need when they want to validate tool behavior without pretending every task has only one valid route. See the Google ADK evaluation criteria for the underlying concepts.

Why answer-only evals miss the real release risk

A response-only eval is useful, but it is incomplete for real tool use. It tells you whether the visible output resembles the reference answer, or whether a judge considers the answer acceptable. It does not tell you whether the agent used the right data source, avoided a prohibited action, preserved required arguments, or followed the workflow that product and operations agreed to support.

Consider a support agent that can inspect subscription status, check recent invoices, and draft a credit request. The final message to the customer might be polite and accurate. Still, the release should fail if the agent skipped the invoice check before drafting the credit, called an internal write API when only a read was allowed, or invented a policy reason that was not present in the retrieved context.

This is why trajectory belongs beside answer quality. The output is what the user sees. The trajectory is what the business must trust.

The same pattern appears in adjacent AI product work. AI evals need fail-case budgets because a single average score hides the risks that matter most. For agents, the fail case is often not a bad sentence. It is an unacceptable sequence of tool calls.

What belongs in a trajectory gate?

A useful trajectory gate has five parts.

First, name the risky task. Do not start with the entire agent. Pick the task where a wrong path creates the highest cost: refund approval, account change, compliance classification, production deployment, lead routing, medical intake, or finance reconciliation. The gate should be narrow enough that the team can write concrete examples.

Second, define the expected tool path. This does not mean recording hidden chain-of-thought. It means listing observable operations: fetch account, retrieve policy, compare eligibility, request confirmation, create draft, log decision. The path should use the same nouns that appear in traces and logs, so engineering can test it and product can review it.

Third, define tolerated variance. This is where teams often resist gates because they fear deterministic testing will make the agent less useful. The answer is not to avoid trajectory checks. The answer is to choose the right strictness. Some paths require exact matching. A payment reversal may need the same tools, in the same order, with no extras. Other tasks need in-order matching. A research assistant may perform extra searches, but it must retrieve evidence before producing the recommendation. Other tasks can allow any-order matching. If several enrichment calls are independent, the release gate can care that they happened, not which one happened first.

Fourth, define response and evidence acceptance. A trajectory gate should still check the answer. The final response should satisfy the task, cite or reference the context it used where appropriate, and avoid unsupported claims. For retrieval-heavy agents, groundedness is not decoration. It is the difference between using context and laundering a guess through fluent prose.

Fifth, define safety and regression thresholds. Safety should cover both the final response and tool behavior. Regression should compare the new build against known risky cases. A release should not pass because the happy path improved while a known edge case silently broke.

How strict should the gate be?

The gate should be strict where the business process is strict, and flexible where the user problem is flexible.

Use exact trajectory gates when the task has a canonical operational sequence. Examples include identity checks before account changes, policy retrieval before entitlement decisions, human approval before irreversible actions, and validation before write calls. In these cases, variance is not creativity. It is risk.

Use in-order gates when the task allows exploration but still has mandatory milestones. A sales research agent may call several enrichment tools, but it should not draft the outreach rationale before it has checked the account record and the latest interaction history. A coding assistant may inspect files in different orders, but it should run the agreed tests before claiming the change is ready. This connects to the same control logic behind AI-assisted coding needs ambiguity checkpoints: flexibility is valuable only when the checkpoints are explicit.

Use any-order gates when the sequence is not material. If three read-only sources all contribute to the same eligibility view, the release gate can accept any order, provided all required calls appear and no forbidden calls appear.

The practical rule is this: do not gate the model’s private reasoning. Gate the observable commitments that affect users, data, cost, compliance, or downstream teams.

Where does this sit in the release ritual?

A trajectory gate should sit before rollout, not after incident review. Put it in the same release conversation as acceptance criteria, analytics, observability, permissions, and rollback.

For a new agentic feature, the PM and tech lead should write one gate together. The PM names the task, user promise, business risk, and unacceptable outcomes. The tech lead maps the expected tool path, trace fields, match type, fixtures, and threshold. If the feature depends on permissions, the gate should confirm that the agent cannot reach tools outside its role. That operational boundary is close to the argument in AI agents need tool permissions, not blanket access: an agent should be evaluated inside the authority it is allowed to exercise.

Then make the gate visible in the release review. A good review can answer these questions without improvising:

• What is the riskiest task this release enables?
• Which tool calls must happen?
• Which tool calls are forbidden?
• Which parts of the path can vary?
• What answer-quality threshold must pass?
• What groundedness or evidence check must pass?
• Which known regression cases were rerun?
• What runtime signal will tell us the gate is failing in production?

That last question matters because pre-release gates are not runtime guardrails. They reduce the chance of shipping a flawed behavior, but production still needs monitoring, traces, queues, escalation lanes, and kill switches. A trajectory gate is the contract you test before release. Runtime controls are how you enforce and learn from that contract after release.

Write one gate before the next agent release

Do not try to evaluate the whole agent at once. That is how teams end up with a spreadsheet of vague scores and no release decision.

Before the next release, choose the riskiest task and write one trajectory gate. Name the expected tool path. Choose exact, in-order, or any-order matching. Define tolerated variance. Add groundedness, safety, and regression thresholds. Decide what failure blocks release and what failure triggers investigation.

The goal is not to make agents mechanical. The goal is to make flexibility releasable. If the agent can take many valid paths, say which paths are valid. If some steps are mandatory, make them visible. If some tool calls are unacceptable, fail the build before users discover the boundary for you.