AI Agent Acceptance Testing NZ

Acceptance testing is a business workflow test, not just a technical test

A technical test can show that an integration runs, a prompt returns an answer, or a tool call completes. Acceptance testing asks a wider question: can the business responsibly use this agent in the workflow it was built for?

That means the reviewer checks whether the agent uses approved sources, follows the operating instructions, avoids unsupported claims, respects permissions, and stops before consequential actions. It also checks whether the human reviewer can understand and approve the prepared work quickly enough for the workflow to be useful.

Why demo success is not enough for live work

Demos usually use clean examples. Live work brings incomplete CRM fields, old files, vague customer messages, conflicting instructions, missing context, unexpected requests, staff changes, and stale source material.

A useful demo is a starting signal. Acceptance testing is the decision gate. It should include normal examples, edge cases, bad inputs, and situations where the agent must stop.

1. Test the real workflow from trigger to review

Start with the actual event that will trigger the agent: a website enquiry, inbox message, CRM stage change, weekly report, uploaded file, meeting note, or content request. Follow the work through to the human review point. Check whether the agent prepares the right output, puts it in the right place, and makes the next human decision easier.

2. Confirm the agent uses approved source material

List the sources the agent is allowed to use: offer pages, proposal templates, CRM fields, SOPs, knowledge-base articles, reports, folders, or approved public pages. Then test whether its output can point back to those sources. If the answer depends on unofficial notes, memory, or guessed context, the source model is not ready.

For source-quality preparation, use the AI Agent Implementation Plan NZ and the AI Data Readiness Checklist for NZ Businesses.

3. Check permissions before connecting CRM, inbox, files, or website tools

Acceptance testing should separate read, draft, recommend, update, send, publish, delete, and admin permissions. Read-only or draft-only access is usually the safer starting point. The AI Agent Permissions Checklist NZ gives a practical permission map before live tool access expands.

4. Sample outputs against tone, accuracy, and claim boundaries

Review a sample of outputs across normal and messy cases. Check whether the agent uses the right tone, avoids over-promising, names uncertainty, and stays inside approved offers and prices. For public pages, sales material, proposals, or customer replies, check claim safety before speed.

5. Prove the approval gates stop consequential actions

The agent should pause before emails, DMs, booking handoffs, pricing, quotes, finance actions, public publishing, legal or privacy wording, sensitive CRM changes, and actions that affect trust. Use the AI Approval Gates for Business Automation guide to decide where a person must approve, edit, reject, or escalate.

6. Test exceptions, bad inputs, stale data, and missing context

Do not only test the happy path. Give the agent incomplete records, contradictory instructions, stale files, missing fields, unusual customer situations, and requests outside scope. A reliable agent should either ask for review, explain what is missing, or stop. It should not fill gaps with confident guesses. For a more detailed operating model, use AI Workflow Exception Handling NZ.

7. Confirm logs, rollback, and owner review

Before launch, decide what the agent logs, who reviews the logs, how mistakes are corrected, and how the workflow is paused. Acceptance testing should confirm there is a named owner who can review outputs, handle exceptions, and make the stop-or-go decision. After launch, use the AI Agent Monitoring Checklist NZ to keep the workflow reviewed.

Bring real examples

Useful examples include recent enquiries, CRM records, support messages, proposal drafts, reports, content briefs, meeting notes, or process screenshots. Remove unnecessary sensitive details before sharing.

Name the current decision points

List where a person currently checks accuracy, tone, price, risk, timing, customer context, or commercial fit. Those are likely approval gates or AI agent escalation rules.

List the systems involved

Name every tool the workflow touches: CRM, inbox, files, website CMS, spreadsheet, calendar, project system, accounting tool, reporting dashboard, or internal knowledge base.

Define what “ready” would mean

A practical go-live standard could include: approved sources are current, reviewer can understand the output, risky actions are blocked, exceptions are escalated, logs are visible, and a human owner accepts the launch decision.