Start with Worlds
The versioned commands below use the published 0.9.0-beta.5 package. The release page describes its redesigned dashboard, starter navigation and existing scope boundaries. Earlier public versions keep their original behavior. Review storage and upgrades before changing versions.
Test your agent against local copies of the systems it uses. Give it a task, let it act, and check what changed in the records. The same test can run on your laptop and in CI.
Choose your path
| Start here | What you will do |
|---|---|
| Use the dashboard | Describe a workflow, review its test, run an agent and inspect results |
| Present the included workday | Run the complete prepared example from a fresh install without an API key |
| Connect your agent | Configure task input and the local endpoints your SDKs use |
| Use the CLI | Set up a project and run a repeatable test from your terminal |
Need the app first? Get started provides the current beta installation instructions. You need Node 24 or newer. You can explore the dashboard before connecting an agent.
The testing loop
- Describe the workflow. Add an optional policy or supported starting records, or open the included example.
- Review the proposed test. Check its starting situation, work to do and success rules, including changes the agent must avoid.
- Save and run. Choose a reusable agent connection. The runner creates fresh isolated worlds for the saved test.
- Understand and improve. Inspect failed expectations and captured changes, then fix the agent and run again with the same saved inputs.
The workflow is shared across connectors. The current beta includes Stripe, native Zendesk and combined support-and-billing workflows within their documented scope. Stripe has dated live API checks; native Zendesk's live API qualification is pending.
A few useful terms
| Term | Meaning |
|---|---|
| World | An isolated environment with its own records, credentials and clock |
| Seed | The starting data used to create or reset a world |
| Task | What you ask the agent to do |
| Outcome | A record change you expect, or a change you prohibit |
| Suite | Tasks, starting data and grading rules saved together |
| Scenario | Repeatable failure conditions, such as rate limits or API errors |
| Diff | The records created, updated or deleted during the run |
For example, a support task might require a ticket to be assigned to a particular group. A billing task might require one refund of the correct amount. Both should also protect unrelated records.
CLI quickstart
The example below uses the bundled Stripe billing tasks. For native Zendesk or combined suites, follow Zendesk test files. The dashboard guide covers all supported workflows through the browser.
Read the result
Start with the overall status, then inspect failed expectations and unexpected changes. A convincing reply is not evidence that the correct record changed.
Tasks need explicit outcomes to establish correctness. A task without an expected answer can still be checked against shared protections, but should not be treated as a proven success. The optional AI judge adds policy commentary; its verdict is advisory and does not change the automated pass/fail result.
See result statuses for exit codes, or dashboard results for the review and replay workflow.
Troubleshooting
| Problem | Next step |
|---|---|
| No requests reached the world | Check the agent command, working directory and SDK settings |
| Requests arrived, but an operation is unsupported | Check connector scope and the failed request |
| A task was omitted during import | Review requester matching and column mappings in Tasks and outcomes |
The result says policy_failed |
Read the failed outcome or invariant and agent exit status; editing the advisory judge's policy alone will not change it |
FAQ
Which connectors can I use? The beta.5 includes scoped Stripe, native Zendesk and combined workflows, plus the documented experimental Agentforce Runtime, SAP S/4HANA, Dataverse, Confluence and Freshservice subsets. JSM has state foundations without native endpoints. Browser/storefront workflows and all three starters have separate scope boundaries. The connector reference distinguishes included subsets, qualification and future catalog requests; inclusion does not imply complete provider compatibility or support in every dashboard workflow. Review the release notes before upgrading an existing project.
Does my key get stored anywhere? The Stripe account import uses its source key in memory and does not write it to disk. Use a restricted read-only key (rk_) or a test-mode key. Dashboard setup and worlds init refuse live secret keys (sk_live). Agent profiles can separately reference private environment files for your own credentials.
Do I need my own data? No. Start with a bundled example, explore its records and define a small test. Import your own data when you are ready.
Can I run this in CI? Yes. CLI setup generates a test harness and workflow. The world starts inside the job; see Deployment and CI.
Does anything leave my machine? Tests, customer records and evidence stay local. The beta.5 offers optional aggregate usage reporting, with visible controls and an opt-out; older published packages retain their original behavior. Existing users and unknown CLI/CI installations stay off until they choose. Installation, explicit source imports, configured webhooks, your agent and optional model features can contact their configured services. Website signup is separate. See privacy.
More detail
Use Connect your agent for SDK and task-input examples, Tasks and outcomes for file formats, and Failure conditions for retry and error tests. After signup, the CLI setup page can generate setup instructions for a coding agent.
Licensing
Free beta use covers internal team testing and CI. The package includes the full terms in LICENSE.txt; see beta terms.
How Worlds differs
Worlds supplies a local, stateful test environment and grades allowed changes and prohibited side effects. Trace and evaluation tools can complement that evidence; their capabilities vary by product. Conversation simulators exercise dialogue, while Worlds exercises implemented business-system workflows. Compare current vendor documentation for the specific requirements your team has.
Worlds provides deterministic replay, controlled failures, a logical clock and scoped connector behavior. Stripe drift evidence is dated and published with gaps. Native Zendesk has local conformance coverage; live API qualification is pending. Combined workflows inherit both connectors' limitations. Free beta access permits internal team use and CI without metering runs.