Every business has rules, even the unwritten ones
In short: your team's rules live in three places: written policies, the systems you use, and people's heads.
Written instructions are usually the only rules anyone hands the model directly. The unwritten ones are the rules it breaks most quietly.
Draft every rule with the person who knows the work, decide which ones should become policy, then turn each approved rule into a test.
Why does your AI agent need your rules at all?
A new support agent, human or AI, can be polite and quick on day one. What they can't know is how your business actually runs: who gets a refund without a manager, which accounts need a second look, when a case must go to a person.
A human hire picks this up over weeks. They read the handbook, sit next to a colleague, get corrected, and remember. An AI agent does none of that. It follows what it was given, at full speed, across every conversation at once.
So the rules matter a great deal, and the question that comes first is where they come from. Most teams answer "the policy document". That is only part of the rule book.
Where do your business rules actually live?
In most operations, rules come from three places.
1. Written down. Policies, standard operating procedures, help-centre articles, training decks, the approval matrix in a spreadsheet. This is the part most teams hand to an AI agent, and it is usually longer than people expect. When we split one public telecom support policy into single rules, one instruction each, it came to 51. It also makes a real difference on its own: in our telecom test, adding the written policy alone raised the share of required actions the agent completed by a wide margin (Anatomy of an AI agent).
2. Built into your systems. Some rules are never written in a policy because the software enforces them. The refund screen won't accept an amount above the limit. The order system won't ship to an address that failed a check. A plan change needs a field that only a supervisor can fill in.
These rules are easy to lose, because the person using the screen never has to think about them. An agent does meet some of them through its tools and permissions: if the system behind the screen enforces the limit, the agent hits it too. But an agent connected straight to that system skips the screen entirely. If the limit lived only in the screen, the agent never meets it.
3. In people's heads. "We never send a refund to an account whose email changed yesterday." "If a business customer calls about an outage, check their other sites too." "Don't offer the retention discount to anyone who's had it in the last year." Every experienced team has dozens of these. They are learned by doing the job, and they are rarely written down because everyone already knows them.
Why do unwritten rules break AI agents?
Because nothing tells the agent they exist, and nothing tells you it broke them.
Take the email rule above. A customer writes in: their parcel never arrived, and they want a refund to a new card. The written policy says refunds for undelivered orders are allowed. The agent checks the order, sees it was never delivered, and issues the refund. The reply is friendly and correct by the policy it was given. What it missed is that the account's email and card were changed three hours earlier, which is exactly the pattern your fraud team would have caught.
A reviewer reading the transcript would likely approve it. The mistake shows only in what changed in your systems: a refund went out that your team would have held.
That's the pattern with unwritten rules. The agent isn't careless. It simply wasn't told, and it doesn't know to ask.
How do you get rules out of people's heads?
Asking someone to "write down all the rules" rarely works. People don't remember rules in the abstract; they remember them when a case calls for them. So start from cases.
Start from real work. Sit with the person who knows the operation best and go through recent cases, especially the awkward ones: escalations, complaints, refunds that were refused, cases a supervisor overturned. For each, ask "why did you do it that way?" The answer is usually a rule.
Let AI do the drafting. An AI model is good at drafting candidate rules from your policies, help articles and case notes. It is not the right judge of which ones are true for your business. Use it for the first draft, not the final word.
Write each rule the same way. A useful rule says when it applies, what to do or not do, and what the exceptions are. For example: "If the account's email or payment card changed in the last 24 hours, do not issue a refund. Hand the case to the fraud team." Note where each rule came from: the policy, a system, or a person.
Have the expert approve every rule. This is the step that turns a draft into a rule book. The person who owns the operation reads each rule in plain language and marks it approved, changed or wrong. Approving a list is much faster than writing one, and it puts the decision where it belongs: with the person who knows the work.
Decide which rules should become policy. Finding a rule is not the same as deciding it should be one. A habit that an experienced person follows may be out of date, may differ from what a colleague does, or may never have been authorised. Capture it, then decide. For rules with real consequences, such as refunds, fraud holds or anything with a legal or regulatory edge, the expert's approval is the start: the person who owns that risk, in finance, fraud or compliance, signs off too.
The result is one approved rule book that holds all three sources. That rule book is what goes into the agent's instructions and its checks, and it is what you test against.
How do you turn a rule into a test?
Each approved rule becomes one or more tests: usually one for the normal case, plus tests for its boundaries and its exceptions. A complete test has four parts:
- Setup: the starting state of your systems that makes the rule apply. Here: an undelivered order, and an account whose email and card changed three hours ago.
- The request: what the customer says. "My parcel never came. Please refund it to my new card."
- Expected end state: what your systems should look like afterwards. No refund issued; a case open with the fraud team.
- The rule it checks: the approved rule this test is about, so a failure points straight to the rule that was broken.
Because the rule book records where each rule came from, a failed test tells you which source it traces back to: the policy, a system, or a person's experience. For what makes a test check its rule properly, see Seven ways your AI agent's test results can mislead you.
This is what AI agent testing should mean for an operations team: not "does it sound good?" but "does it follow the rules we approved, in the cases where they matter?"
What about agents with few hard rules?
Not every agent works in a world of limits and approvals. A research assistant, a writing assistant or an internal knowledge agent has fewer rules of the "never refund above $100" kind. It still has rules. They just look different.
Ask the expert the same question: what makes a piece of work right or wrong here? The answers are usually rules about quality and boundaries:
- Every figure must come from a source the agent can name.
- Quotes must match the source word for word.
- Never mention clients by name in external drafts.
- Stick to the brief's length and audience.
Some can be checked with code, which gives the same answer every time: does every quote appear in the source, does a client name appear in the text? Others need judgement, such as "is this summary faithful to the report?" For those, use an AI judge, but give it the exact rule for each test rather than asking whether the work is "good", and have a person review a sample of its verdicts. The rule book still comes first; only the way you check changes.
Where should you start?
Pick one workflow your agent handles or will handle. Then:
- Collect the written policy for it.
- List the checks your systems enforce that the agent might bypass.
- Spend an hour with your most experienced person on recent awkward cases, and capture the unwritten rules.
- Approve one rule book, then write tests for each rule, covering its normal case, boundaries and exceptions, starting with the rules that cost the most when broken.
That is how you test your AI agent against your own rules, not against someone else's idea of good service.
If you'd like help with step 3, we run rule-discovery sessions: we sit with your team on one workflow, work through recent cases, and leave you with a drafted rule book ready for approval. Write to hello@constolve.com to set one up, or see what we test at app.constolve.com.
Questions people ask
What are business rules for an AI agent?
The instructions that say what the agent may and may not do in your operation: limits, approvals, checks, hand-offs and exceptions. They come from written policies, from your systems and from your team's experience.
Can I just give my AI agent our policy documents?
It's a good start and it makes a real difference, but it's rarely enough. Rules your systems enforce and rules your team follows from experience are often missing from the documents, and the agent can't follow a rule it was never given.
How do I capture unwritten rules?
Go through recent awkward cases with the person who knows the work and ask why each was handled the way it was. Let AI draft rules from the answers and your documents, have the expert approve each one, and have the owner of the risk sign off on the consequential ones.
How do I test an AI agent against our SOPs?
Turn each approved rule into a test with a setup that makes the rule apply, a customer request, the expected end state of your systems and the rule it checks. Check what changed in your systems, not only what the agent said.
How do I test an agent that has few hard rules, like a research assistant?
Write down what makes its work right or wrong, such as named sources, exact quotes or banned names. Check what you can with code, and use an AI judge given the specific rule for the rest, with a person reviewing a sample.