How to test AI agents before production

Start with a concrete task and the state it is allowed to change. A useful test can explain the expected result before reading the agent's reply.

Choose a supported workflow

The Worlds public beta includes Stripe, native Zendesk and supported combined workflows in the local dashboard and CLI. Read the current limitations first. Stripe's implemented surface has real-API evidence; native Zendesk live qualification is pending. Other catalog entries collect interest for future connectors.

Prepare representative starting data

Use a reviewed seed or import within the connector's supported scope. Native Zendesk uses supported offline snapshots and reviewed Full JSON exports. Keep credentials, identity maps and private customer text out of shared reports and files. The dashboard can help you review the records before creating a world.

Draft the test with AI

Describe the request your agent should handle and add the policy it must follow. Optional AI drafting can propose the starting situation, tasks and success rules. Review and approve the inputs before sending them to your configured OpenAI or Anthropic provider. Manual authoring is also available.

The draft is a proposal for you to inspect and edit. Check its assumptions, records and account relationships before accepting it. Configuring a drafting provider does not connect the agent you want to test; that agent still needs its own program and connection settings. See the dashboard guide and the beta's drafting limitations.

Review expected changes and protections

Say which actions should happen and which objects must remain unchanged. For example, a correct ticket update must preserve the requester and respect comment privacy. A refund expectation must identify the intended billing record. Supported combined tasks need explicit links between the relevant customer and ticket.

Connect, run and inspect

Connect a local command using the documented task and endpoint contract. Run a fresh test, inspect authoritative outcome checks and explain any ungraded task separately. Optional model judgments do not replace state assertions. Correct the agent and repeat with the same saved inputs.

Reuse the test

Preserve a named setup and suite for a later code change or internal CI run. Reruns reuse those reviewed inputs without asking AI to draft them again. Add a supported failure condition to test recovery, or advance a retained world's clock to inspect later effects. Clock exploration does not change the original recorded grade.

A scripted sample helps you learn the controls, but does not establish performance for your own agent. Your model provider and dependencies may still need network access even when Worlds runs locally.

Join the beta for the installation guides. Read transcript and outcome grading to understand the evidence each provides.