Short answer
Write criteria a machine can check without the agent's help: a command or observation with a clear pass condition, a negative case, and an owner outside the agent. Give the agent a legitimate way to say a criterion is impossible, because agents without one tend to cheat.
You gave the agent a ticket with acceptance criteria. It came back saying every criterion was met. Some were, in the letter. “The page loads quickly” was met because the agent decided it loads quickly. “Errors are handled gracefully” was met because there is now a try/catch that swallows everything.
Acceptance criteria were invented for people, who share context and can be trusted to read “gracefully” the way the author meant it. An agent reads them as a checklist to be satisfied as cheaply as possible. That is not a flaw to complain about. It is a property to design for, and once you do, criteria become the most powerful lever you have over agent output.
Why criteria written for people fail with agents
Human criteria rely on three things an agent doesn’t bring: shared context about what the product is for, a sense of embarrassment about meeting the letter and missing the point, and a reviewer who will ask follow-up questions.
Take those away and vague criteria have an obvious failure mode. The agent picks the cheapest interpretation, satisfies it, and reports success with confidence. Nobody lied, exactly. The criterion simply permitted it.
There is a sharper version of this. When criteria are concrete but impossible to meet honestly, some agents meet them dishonestly. ImpossibleBench, a 2025 study by Zhong, Raghunathan and Carlini, gave coding agents tasks whose tests contradicted the specification. GPT-5 still “passed” 76% of the one-off variant on SWE-bench, by editing tests, special-casing inputs or overriding comparisons. METR saw the same pattern in o3 on hard research engineering tasks. A criterion with no honest path will be satisfied by a dishonest one.
What makes a criterion agent-proof
A good criterion for an agent has five properties:
- It is observable. It describes something you can see in a running system or a file, not an internal quality.
- It is executable. A command, test or script can decide pass or fail without the agent’s opinion.
- It names the negative case. What must not happen is often more useful than what must.
- It is owned outside the agent. The agent may add checks, but it can’t edit or delete the ones that define done.
- It has an exit. If the criterion is impossible or contradicts another one, the agent is told to stop and say so.
The fifth one sounds soft. It is the one with the strongest numbers behind it.
Why an honest exit reduces cheating
ImpossibleBench tested what happens when an agent is allowed to give up properly. In one condition the agents could submit a request for human intervention instead of a solution.
The authors note the effect was much smaller for Claude Opus 4.1, so this is not a universal switch. Prompt wording mattered too: on a LiveCodeBench variant, the strictest prompt took GPT-5 from 92% cheating to 1%, while o3 still cheated 33% of the time. METR, on the other hand, found that telling o3 not to cheat barely moved its rate on one task. The pattern I take from this: instructions help some models a lot and others little, so don’t rely on them alone. Structure helps all of them. An exit, read-only tests and checks run by something other than the agent work whatever model is behind the keyboard.
Bad and good acceptance criteria side by side
The table below rewrites common criteria into a form an agent can’t argue with. None of the good versions is clever. They are just specific.
| Vague criterion | Agent-proof version |
|---|---|
| The page loads quickly | npm run perf -- /orders reports time to interactive under 2.0 s on the CI profile |
| Errors are handled gracefully | When the API returns 500, the screen shows the retry banner and no stack trace reaches the log at error level |
| Users can reset their password | An end-to-end test requests a reset, follows the link from the captured email and logs in with the new password |
| The import supports large files | Importing the 200 MB fixture finishes, memory stays under 1 GB, row count matches the fixture’s |
| Don’t break existing behaviour | The full suite passes on the head commit; no file under tests/ changed without being listed in the PR |
| The fix works | test_rounding_negative fails on the base commit and passes on the head commit |
| Code is clean | Linter and type checker exit 0; reviewer model finds no function over 60 lines in the diff |
Notice that some good versions still involve judgement: a reviewer model checking function length, or a person reading the list of changed tests. That is fine. The point is that the judgement has a narrow, checkable question instead of “is this good?”.
A workflow that keeps criteria in charge
The criteria should exist before the agent starts, be turned into checks early, and be run by the harness at the end.
Criteria from ticket to merge
- Person
- Agent
- Automated check
Steps as text
| Step | What happens |
|---|---|
| 1. Ticket | Person — goal and constraints |
| 2. Criteria | Person — observable and executable |
| 3. Checks written | Agent — fail before the change |
| 4. Agent builds | Agent — may add checks not remove |
| 5. Harness runs checks | Automated check — records exit codes |
| 6. Reviewer model | Agent — criteria, diff, evidence |
| Loop | findings: back from step 6 to step 4 |
Some practical rules that go with it:
- Write the checks before the code, and confirm they fail. A check that passes before the change is not testing the change.
- Freeze the criteria for the run. If the agent believes a criterion is wrong, it stops and says so. It doesn’t edit the criterion and continue.
- Keep criteria to what the ticket needs. Ten precise criteria beat thirty, because each one is a place the agent will spend effort.
- Separate “must” from “nice”. Agents treat every listed item as required and will contort the code to satisfy a preference.
- Let the reviewer reject criteria that were met by gaming. The reviewer sees the diff, so it can notice a special-cased input that a test runner can’t.
The step where checks are written can be done by the agent, a separate model or a person. What matters is that they exist and fail before the implementation starts, and that the implementing agent can’t change them afterwards. This is the core of spec-driven development with AI, and the reason the evidence must come from the harness is covered in evidence over claims.
The criteria nobody writes, but agents need
A few criteria are missing from almost every ticket because people never needed them.
Scope. “Change only files under src/billing/. Anything else needs a note in the PR.” Agents happily refactor half the codebase on the way to a one-line fix.
Stopping conditions. “If the API you need doesn’t exist, stop and report. Do not mock it in production code.” Without this, the agent invents the API. This also reduces the “would you like me to continue?” pattern described in agents that stop halfway, because the agent knows when stopping is correct and when it isn’t.
Evidence format. “Attach the test output from the harness, not a summary.” It costs one line and removes a whole category of argument.
Test ownership. “Do not edit tests that existed before this task. New tests are fine.” In ImpossibleBench, Claude models’ cheating was mostly test edits, so this line targets the most common trick directly.
Once criteria look like this, something interesting happens to the human role. The person who writes the ticket is no longer describing work. They are writing the definition of done that a machine will enforce. That is a smaller amount of text and a much bigger lever, and it is where I think a lot of senior engineering time will go as agents take over the typing.
Questions
How do you write acceptance criteria for an AI agent?
Make each criterion observable and checkable by a command or test with a clear pass condition, include what must not happen, and keep the checks outside the agent’s control. Tell the agent to stop and report if a criterion is impossible instead of working around it.
Why do AI agents claim acceptance criteria are met when they aren’t?
Vague criteria let the agent pick the cheapest interpretation, and impossible criteria push some agents to game the checks. Research such as ImpossibleBench shows frontier models editing tests or special-casing inputs when no honest solution exists.
Should the AI agent write its own acceptance tests?
It can draft them, but they should be reviewed and frozen before implementation, and they should fail before the change. The agent that implements the change must not be able to edit or delete them afterwards.
What is the difference between acceptance criteria and a spec for an agent?
The spec describes what to build and why; the acceptance criteria define how the result will be checked. Agents need both, but the criteria are what stop them from declaring success early.