Ancha build plan

Make the other side a one-command job for coding agents.

Version 1 · 2026-10-30 · Product owner: Patrick · For the coding agent: read /spec.md

Work order for the coding agent that builds Ancha. Version 1, 2026-10-30. Product owner: Patrick. Written by Claude (session "ancha-product-strat") on Patrick's direction. Plain Markdown of this page: https://ancha-build-plan.pages.dev/spec.md

0. Read this first

You are the engineer on Ancha. This page is your work order from the product owner. Follow it in order. Where it conflicts with older plans in the repo (docs/HANDOFF.md, the plan14 ledger, cloud/NEXT.md), this page wins. Where it conflicts with a newer instruction from Patrick in chat, Patrick wins.

• Ownership changed on 2026-10-30. Jay no longer manages the product. Patrick owns product decisions. You own all of the code: packages/, backend/, frontend/, cloud/, scenes/, examples/, docs/. cloud/JAY.md is history; don't write hand-off notes for Jay.
• Repository. ~/real_ancha is the repo. Branch main is what ancha.io serves; it has never been pushed to origin. ~/real_ancha/cloud is a separate git repository with the live hosted code and nothing committed yet. ~/real_ancha-anthropic is a website worktree; leave it alone.
• Read order. §1 (the point), §2 (product shape), §3 (rules), then your phase. §15 lists decisions you must not make yourself.
• Every phase ends at a gate. Stop, report to Patrick (§3.6), and wait for his go-ahead or new orders.
• Start here: §5.1 step 1.

1. The point

1.1 The value test

Patrick, 2026-10-30, verbatim:

关键是要抓住一个点 也就是你不能亲自做的事情 或者比如 我让你做一个agent。 这个ancha 要提供的价值必须《= 你整个任务如果这一部分你自己做 占据全任务的30%。 也就是要从 30% 缩成3% 最好还是你自己完成不了的。

In English: when a coding agent builds an agent for someone, Ancha must take over a part of the job that would cost at least 30% of the whole task if the coding agent did it itself, and shrink it to about 3%. Best of all, it should be a part the coding agent cannot do by itself at all.

Judge every feature in this plan by that test. A feature that doesn't move a coding agent's work from about 30% toward about 3%, and doesn't do something the coding agent can't, waits.

1.2 Where the 30% is

The 10-28 lab. Claude built a billing-support agent and tested it the way a developer at a polished agent company would. The code is in ~/ancha-strategy/lab/ and the findings in its NOTES.md.

• The test infrastructure was about three times the agent's own code.
• The fake of the agent's own six tools took minutes. The lab's verdict: "What I would not have paid for: a fake of my own six tools."
• The people broke everything. Three parties broke the agent: its answer to the manager went to the customer; it never asked the customer the manager's question (0 of 3); it failed to relay an answer once.
• They also broke the test harness seven ways, and each one made a result lie:
  1. the simulated customer's stop token ([DONE]) cut off the agent's turn;
  2. the manager was built from the customer's template;
  3. a decision was read from a text prefix, so "APPROVE or DENY? Before I decide…" was recorded as approved;
  4. a typo ("denyd") in the harness's own state;
  5. the customer revealed its hidden fact unprompted, 3 of 3 times, which hid that the agent never asked;
  6. an LLM judge's reasoning said pass while its JSON said fail;
  7. outcome-only assertions passed while an approval was stuck.
• Cost: a three-party trial took 90–110 s and $1.13–$3.61 of model time. The fake world itself took about 0 ms.

Accounts. On 2026-10-30 a coding agent could not sign in to our own demo account: coding agents can't create accounts or type passwords, by policy and because of bot detection. The same holds for Slack workspaces, PagerDuty, Datadog, GitHub orgs, Salesforce sandboxes and Stripe. The official MCP servers for Slack, PagerDuty and Datadog are hosted services behind OAuth (secondary sources, checked 2026-10-30), so an agent built on them can't be pointed at a local mock by changing a base URL.

Fresh-user QA (session ancha-qa, 2026-10-31 UTC). With today's CLI, a new user who already has a world gets from ancha init to a planted business bug caught and a fix proven in about three minutes. The loop works once the world exists. Building the world is the cost.

1.3 The point, in one line

A coding agent can write the agent, and a mock of its own tools. It can't build the other side.

The other side is everything the agent deals with that the coding agent doesn't control:

• systems it can't open: other companies' tools, which need accounts and can't be reset or broken on purpose;
• people it can't play reliably: customers, teammates and approvers, each with their own facts, decisions and timing;
• time it can't wait for: an approval two hours later, a reply the next day, a deploy that lands after CI;
• incidents it can't cause: an outage, a timeout after a commit, a 429;
• a verdict it didn't write: a coding agent that grades its agent with its own mocks is grading its own homework.

Ancha builds the other side on demand and grades the agent against it. The coding agent writes what the agent must achieve and who is involved. Ancha does the rest.

1.4 The job, before and after

1.5 The metric

North star: the share of a coding agent's effort that goes into setting up tests when it builds an agent.

• Baseline without Ancha: expected at least 30%.
• Target with Ancha: at most 3%.
• Guard 1: the agent it ships passes a hidden evaluation at least as often as without Ancha.
• Guard 2: Ancha produces zero lying results, meaning verdicts that disagree with ground truth.

§10 defines the benchmark. Build it in Phase 0 and rerun it at Gate 5 and Gate 9.

1.6 What we are not building

• A twin-count race. As of 2026-10-28, Arga listed 46 twins, Archal 22 environments, and Rystic 17 simulators, each with a published match rate against the live API. LocalStack reportedly bought WonderTwin. We build twins only for the reference worlds, and publish fidelity numbers for those.
• A general eval platform. Braintrust and LangSmith own datasets, experiments and dashboards. Our console shows what only Ancha has: the world, the people and the process.
• "A simulated user." One LLM user is free in Google ADK and LangWatch Scenario, and polished support companies build their own. Our unit is a company of people acting through real tools over time.
• New marketing claims. Don't edit ancha.io unless Patrick asks (§3.2).

2. Product shape

2.1 Users

• The operator is the engineer's coding agent (Claude Code, Codex, Cursor). It does everything through the CLI, with --json.
• The approver and reader is the engineer. They approve ancha login once in the browser and read results in the console through links the CLI prints.
• CI uses an API key and gates pull requests on the pass rate.

2.2 The journey

This sequence is the product. Commands that don't exist yet are built in the phases below. Names change only with Patrick's OK.

# The engineer tells their coding agent: "test our support bot with Ancha"
npx @ancha-labs/cli init       # detects tools; writes ancha.yaml, a world, a test, the coding-agent skill; ignores .ancha/
ancha login                    # the engineer approves once in the browser (skip it to stay local)
ancha test new "Laura says her CSV exports are empty; only Ben knows the workaround; Mike decides the priority"
ancha run                      # agent on this machine, world in the workspace; prints the pass rate and a console link
ancha explain --json           # why it failed: the check, the conversation, the calls
#   ...the coding agent fixes the agent...
ancha run --on cloud -n 20     # sealed; 20 attempts in parallel on the workspace machine
ancha compare                  # this run against the last good one
pytest                         # the same world as a fixture inside the team's own tests (Phase 7)
# CI: ANCHA_API_KEY=… ancha run --on cloud --min-pass-rate 0.95
# The engineer opens app.ancha.io: every run from every place, with attempts, conversations and checks

2.3 One object model

Retire the second vocabulary: a scene becomes a test with an inline world; a trial becomes an attempt; oracle and null become reference agents; companion and fingerprint stay internal. Old files and flags keep working (§9.1).

2.4 Definitions in the repo, results in the workspace

• Definitions are files in the user's repo, reviewed in pull requests: ancha.yaml, ancha/worlds/*.yaml, ancha/tests/*.yaml.
• Results live in the workspace (local .ancha/ or hosted): runs and attempts, append-only, each pinned to the exact definition hashes and git commit that produced it.
• Ownership goes one way, so nothing conflicts:
  • an object that came from the repo is read-only in the console, with "Copy to edit";
  • an object made in the console belongs to the console until ancha pull writes it into the repo; from then on the repo owns it.
• This is the convention of Supabase CLI (migrations in the repo, dashboard changes pulled back), Cypress Cloud and Braintrust (tests in code, results recorded to a dashboard). Don't invent a sync protocol.

2.5 Where the agent runs

• Worlds always come from the linked workspace, local or hosted.
• Every target uses the same evaluator and records to the same workspace.
• local must say plainly in its output that the agent is not sealed (QA finding E1).

2.6 Workspaces

• Local: ancha ui on 127.0.0.1, with files under .ancha/. Free and offline.
• Hosted: one Fly Machine per user behind app.ancha.io (see cloud/README.md). ancha login links it; CI uses ANCHA_SERVER_URL and ANCHA_API_KEY.
• The same CLI commands work against either (Phase 4). Today the CLI reaches only the hosted twins; agents, worlds, tests and runs on the hosted machine are browser-only.

2.7 The console

The console is the human's window: every run from every source, attempts with a Conversation tab, checks with their evidence, Compare, Run again, and the people on the world page. It is not the operator's tool. The coding agent never needs it.

3. Rules

3.1 You may, without asking

• Read and change anything in ~/real_ancha and ~/real_ancha/cloud.
• Run tests, local servers, Docker, the local console and the CLI.
• Commit locally on branches named plan/p<N>-<topic>, and merge them into a local branch plan/integration. main stays the deployed baseline until Patrick says to merge.
• Create bench/, examples/acme-*, docs/product/ and docs/plan/.
• Spend up to $30 of model credit per phase on people, judges and calibration. Log every spend in the ledger (§3.6).

3.2 Ask Patrick first

• Pushing to any git remote (origin, tbskin).
• Publishing to npm or PyPI. Patrick runs the publish; it needs his one-time code.
• Deploying anything to production: the gateway (fly deploy in cloud/gateway), a tenant image (TENANT_IMAGE), machine updates, Supabase schema or config.
• Creating cloud resources: Fly apps or machines, Supabase projects, Cloudflare projects, domains, DNS records.
• Benchmark runs that would spend more than the phase budget. Send the estimate.
• Any edit to ancha.io (website/). Preview it on localhost first and wait for Patrick's OK, block by block.
• Deleting data anywhere, including archived worlds and tests in hosted workspaces.

3.3 Never

• Print, log, commit or paste secrets: API keys, tokens, passwords, cookies, one-time links. Fake twin tokens in artifacts are fine; real keys never.
• Use or ask for the demo account's password. Patrick sets it with cloud/gateway/scripts/set-demo-password.sh.
• Create accounts on or sign in to third-party services. Fidelity probes run only with tokens Patrick provides as environment variables (§12.2).
• Run Rystic's binaries. Its EULA §4(e) bans use for a competing product and §4(c) bans reverse engineering. Read its public docs only.
• Touch Cloudflare projects other than ancha-world-design, ancha-field-notes, ancha-survey, ancha-admin, ancha-decks and ancha-build-plan. The account is shared with other projects.
• Destroy tenant machines or volumes. Reset hosted data by archiving through the console API.
• Use LLM-default names such as Maya Chen, Priya, Dana, Sam or Northwind. Use the cast: Ben (developer), Rachel (on-call), Greg (database), Laura (customer at Pike Street Dental), Mike (manager). One cast per world.
• Fake a successful response for a call a twin doesn't implement. Report a coverage gap in the vendor's error shape.

3.4 Engineering rules that still hold

These come from the repo's AGENTS.md and still bind you:

• Worlds are deterministic: use SimClock and IdStream, not wall time or random ids. People's model outputs are the one non-deterministic input, and they are recorded (§6.3 k).
• Read ctx.store.db at call time. Snapshot restore replaces the database.
• Schemas are strict. Semantic validation lives in shared core code, so the CLI and the console reject the same input.
• Mutations commit synchronously; event delivery happens after the commit, with exact-byte signatures.
• The CLI is non-interactive, has --json on every command and stable error codes and exit codes, and never puts secrets in artifacts or errors.
• Existing results and configurations stay readable: migrate them or read both formats.
• The console and the CLI call the same services. No UI-only execution path.
• Managed attempts stay isolated and never fall back to host execution.
• When boundaries, schemas, commands or entry points move, update AGENTS.md, docs/ARCHITECTURE.md and docs/CODEBASE.md in the same change.

3.5 Done means

For every item:

1. Unit tests cover the new behavior (pnpm exec vitest run <file>), and pnpm typecheck is clean.
2. The item's acceptance commands ran, and their output is in the ledger.
3. For CLI changes: pnpm build, pack, install the tarball under a fresh temporary HOME outside the checkout, and run the commands there.
4. For console changes: checked in a real browser at 1512 and 390 px wide, with screenshots in docs/plan/shots/.
5. Docs are updated in the same change (after Phase 5, in the single docs source).
6. Nothing is claimed that you didn't run.

3.6 Reporting

• Keep docs/plan/LEDGER.md: date, item, files changed, evidence (command plus an output excerpt), money spent, open questions.
• At each gate, send Patrick a short report in Chinese; code, commands and product names stay English. Say what's done, show the evidence, give the numbers, say what's next and what you need from him. No check-mark lists without the raw output beside them.

4. Phases

• A session is one sitting of a few hours.
• Order matters. Phases 1–3 are the point. Phases 4–5 make it one entry. The rest widens it.
• With two engineers:
  • Track A (engine: Phases 1, 2, 3, 6) works in a worktree ~/real_ancha-engine.
  • Track B (entry: Phases 0, 4, 5, 7, 8, 9) works in ~/real_ancha-entry.
  • Track A owns packages/server/**, packages/core/src/{scene,world}/** and examples/acme-*.
  • Track B owns packages/cli/**, backend/src/{runner,playgrounds,console,kit}/**, frontend/**, cloud/**, bench/** and docs/product/**.
  • Shared files (packages/core/src/index.ts, schema exports) change only in small commits merged into plan/integration at each gate.

5. Phase 0: Baseline

5.1 Housekeeping

1. Save this plan in the repo as docs/plan/BUILD_PLAN.md (from https://ancha-build-plan.pages.dev/spec.md), so it's versioned with the code.
2. Snapshot the hosted code. In ~/real_ancha/cloud, commit everything on branch plan/p0-snapshot with the message "Snapshot of the hosted code as deployed (gateway; tenant image v9)". Don't push.
3. In ~/real_ancha, create plan/integration from main. Leave untracked deck sources (website/deck/) alone unless Patrick says otherwise.
4. Rewrite the top of AGENTS.md:
   • ownership: Patrick owns the product; this plan is the active plan, linked;
   • keep the engineering rules (§3.4);
   • mark docs/HANDOFF.md and the plan14 ledger as history.
5. Copy the fresh-user QA findings into docs/qa/2026-10-31-fresh-user.md. Take them from /private/tmp/claude-501/-Users-liuyizhou/6694c427-f06a-4873-9be0-bdcd23eb111f/scratchpad/qa/findings.md if it still exists; otherwise use Appendix B.
6. Create docs/plan/LEDGER.md (§3.6).

5.2 Benchmark harness and baseline

Build the harness of §10 for task B1 only. Run arm A (no Ancha) and arm B-today (the current CLI: worlds and checks, no people), 3 runs each. Ask Patrick before spending more than the phase budget.

5.3 Gate 0

Report the baseline table for arm A and arm B-today: test-setup share, hidden-evaluation pass rate, lying results, wall time, cost.

6. Phase 1: People and the router

6.1 Goal

A world can contain people who:

• answer through the tools;
• keep their facts until asked;
• make decisions that are recorded as state;
• take time to answer;
• can never end a conversation with a magic token.

The agent is woken by native events, and simulated time skips while everyone waits.

6.2 Step 1: seal the control plane (QA finding E2)

Do this first. Today the control API (/_ancha/*) shares the agent's host and port and needs no auth. From inside an attempt, GET /_ancha/envs lists every live environment, and GET /_ancha/envs/<id>/state returns another attempt's world. People's hidden facts will live in that state.

• Serve twin routes and control routes on separate listeners, or require a control token the agent never receives. Prefer separate listeners if the companion layout allows it, and document the choice in docs/ARCHITECTURE.md.
• Give the agent only the variables of tools in its world. Drop ANCHA_WORLD_SEED and ANCHA_TRIAL_TOKEN; give ANCHA_EVENTS_TOKEN only when the agent's input mode needs it; drop the signing secrets of tools that aren't in the world.
• Error hints must not advertise control paths to the agent.
• Acceptance: probe agents that list environments, read world state, and read another attempt's tokens get 404 or 401 in the local, docker and cloud targets. Add the probes as tests in packages/server/test/.

6.3 Step 2: people in the world file

Add the schema to packages/core/src/scene/schema.ts and packages/core/src/world/schema.ts, with the same strictness and shared validation. Example (the $slack_laura, $slack_eng, $github_ben and similar refs are created by the acme-support seed in Phase 3):

people:
  - ref: laura                     # world-unique; referred to as laura or $person.laura
    name: Laura
    role: customer at Pike Street Dental; uses acme's patient-records product
    goal: Get CSV exports working again before Friday payroll.
    style: brief, friendly, not technical
    accounts:                      # who this person is in each tool; must exist in the seed
      slack: $slack_laura
    listens: [dm, mentions, threads]
    replies_after: 3m..12m         # simulated time per reply, sampled from the attempt seed
    hours: 08:00-18:00 America/Los_Angeles   # optional; replies wait for the next window
    knows:
      - id: workspace
        value: Pike Street Dental
        about: which workspace or account has the problem
      - id: beta_on
        value: turned on the export beta yesterday
        about: recent settings changes or beta features
        match: semantic            # default exact (normalized substring); semantic uses a recorded classifier
      - id: blocked_staff
        value: all 12 front-desk staff use the export for payroll
        number: 12                 # optional; lets decision rules compare
        about: how many people are affected
      - id: payroll_friday
        value: payroll runs Friday and needs the export
        reveal: volunteer          # default when_asked
    can: [slack.post]
  - ref: ben
    name: Ben
    role: developer who owns exports at acme
    goal: Fix real bugs; get the details before digging in.
    accounts: { slack: $slack_ben, github: $github_ben }
    listens: [dm, mentions, { channels: [$slack_eng] }]
    replies_after: 10m..40m
    knows:
      - id: workaround
        value: turn the export beta off and run the export again
        about: a workaround or a fix for empty exports
        needs: [laura.beta_on]     # shared only once Ben knows Laura's beta is on
      - id: root_cause
        value: the beta exporter drops every row when a workspace has more than 50,000 patients
        about: the root cause
        confidential: true         # checks can assert it never reaches a customer
    can: [slack.post, github.comment]
  - ref: mike
    name: Mike
    role: support manager
    goal: Keep priorities honest.
    accounts: { slack: $slack_mike }
    listens: [dm, mentions]
    replies_after: 5m..30m
    decides:
      - id: severity
        options: [p1, p2]
        needs: [laura.blocked_staff]   # without it, Mike asks for it instead of deciding
        choose:                        # deterministic; the first match wins
          - option: p1
            when: { fact: [laura.blocked_staff, gte, 5] }
          - option: p2
        # alternative to choose: guideline: "P1 when five or more people are blocked." (decided by the model, recorded)
    can: [slack.post]

The engine enforces every field below. None of it depends on a prompt asking the model to behave.

(a) Accounts. A person is a user in each listed tool, created by the seed. Unknown refs fail validation.

(b) Routing. An action by the agent reaches a person when it matches their listens:

• dm: a direct message to the person.
• mentions: it mentions the person (<@U…> in Slack, @login on GitHub).
• threads: a reply in a thread the person started or replied in, or a comment on an issue or pull request the person opened, is assigned to, or commented on.
• { channels: [...] }: a top-level message in one of those channels.

The router applies the same rules to decide which of the people's actions reach the agent's bot user. Each delivery is a row in world_inbox(person, source_kind, source_id, sim_at).

(c) Replies. For each person with unanswered inbox items, at sim_now + sample(replies_after) (seeded, then moved into hours), the engine calls the model once with:

• the person's role, goal and style;
• the facts the person knows: their own unlocked facts (d), their volunteer facts, and facts others told them (g);
• the decisions they have made (e);
• what they can see: the DM, thread or issue history visible to them, in order, with authors;
• an output schema { messages: [{ where, text }], actions: [{ kind, args }] }, with actions limited to can.

There are no free-text control tokens. A person may stay silent (empty arrays). The engine writes their messages and actions through the twins as the person's account, so the agent sees native traffic.

(d) Fact gating. Before generating a reply, the engine asks a classifier which locked when_asked facts the latest message to this person asks about, giving it each fact's about text. The answer is a list of fact ids. Only those facts are unlocked and passed to the reply prompt.

• A fact never appears in any prompt before it is unlocked.
• A fact with needs unlocks only when the person already knows everything in needs.
• Every classifier verdict is recorded.

(e) Decisions. When a message asks a person to decide (classifier, recorded), or a rule triggers a decision:

• if the person doesn't know a fact in needs, they ask for it instead of deciding;
• otherwise the decision comes from choose (the first matching when) or from guideline (a constrained model call returning one of options with a reason);
• inside choose, { fact: [<person>.<fact>, gte|lte|eq, <number>] } compares the fact's number, and holds only if the deciding person knows the fact;
• the decision is written to world_decisions(person, decision_id, option, reason, sim_at, message_id) before the reply is generated. The reply prompt includes it, so the text can't contradict it.

Text is never parsed for decisions.

(f) Timing. replies_after is a range of simulated time. Sampling uses the attempt seed, so the same seed gives the same delays. hours moves a reply to the next working window. away and patience can come later.

(g) Knowledge. world_knowledge(person, fact_ref, source, sim_at) records who knows what: their own facts, facts the agent told them, and facts another person told them. A message tells a person a fact when its text matches the fact: exact means a normalized substring of value; semantic means a classifier verdict, recorded. People use the facts they know.

(h) Actions and rules. People act only through can, as twin actions under their own account. Rule actors (world_rules.actors) stay, and their when predicates gain people predicates. Follow the predicate style of WorldPredicateSchema (packages/core/src/world/schema.ts) and packages/server/src/world/predicates.ts.

{ knows: [ben, laura.beta_on] }       # Ben knows the fact
{ decided: [mike, severity, p1] }     # Mike's recorded decision
{ told: [laura, ben.workaround] }     # alias of knows, for checks that read better

(i) Ending an attempt. An attempt ends at the first of these:

1. the test's until predicate holds;
2. the world is quiet (§6.5) and nothing is scheduled before the test's simulated deadline;
3. the real-time timeout passes.

Nothing a person or the agent writes can end it.

(j) Recording. Each model call writes one line to people.jsonl in the attempt's artifacts:

{ "at_sim": "…", "person": "mike", "kind": "reply|gate|decide|match", "model": "…", "prompt_sha256": "…", "input_tokens": 0, "output_tokens": 0, "cost_usd": 0, "output": {} }

The console and explain read it.

(k) Cache and replay.

• Cache key: sha256 of the person's spec, the model, the unlocked facts, the known facts, the decisions, the visible history and the attempt seed.
• --people live calls the model and writes the cache.
• --people cached (the default) reuses a cached output when the key matches and calls the model otherwise.
• --people replay:<attempt-id> uses that attempt's recorded outputs and stops at the first divergence, reporting it.
• The same seed with the same agent behavior gives identical people and an identical end state.

(l) Models. defaults.people_model in ancha.yaml (default claude-haiku-5-5) can be overridden per world or per person.

• People run where the world runs and use that workspace's key (ancha secrets set ANTHROPIC_API_KEY or OPENROUTER_API_KEY). Hosted workspaces get a provisioned key once Patrick enables it (§15).
• In sealed targets, the world's container gets egress only to the people model's host, through the same relay that grants the agent its provider host (backend/src/execution/). The key stays in the world's container, never in the agent's.
• Unit tests use a scripted fake provider. No network in tests.

(m) Secrecy. Person specs, locked facts, decisions and people.jsonl must never be reachable by the agent under test: not through any route, variable or file in its container. Test this the same way as §6.2.

6.4 Step 3: waking the agent

Input modes are set by agent.input in ancha.yaml; the console's Connect form exposes the same choices.

• events (exists): a long-running agent receives native events, either pushed to its URL with the vendor's signature (packages/server/src/events/delivery.ts) or pulled from ANCHA_EVENTS_URL.
• http (exists): a chat endpoint. Each message addressed to the bot is POSTed as a chat turn, with one conversation per thread.
• per_event (new): a run-once command is started again for every event addressed to the bot, with ANCHA_EVENT_JSON (the native payload) and ANCHA_TASK (one line: who said what, where). Invocations are serialized per conversation. This lets simple scripts hold multi-turn conversations.
• command (exists): run once with ANCHA_TASK. People still answer, but the agent sees their replies only if it polls.

"Addressed to the bot" uses the routing rules of §6.3 (b), applied to the bot user.

6.5 Step 4: simulated time that skips while everyone waits

• The world is quiet when, for idle_ms of real time (default 1500), no agent request is in flight, no per_event invocation is running, and no event is undelivered.
• When quiet, advance the simulated clock to the next scheduled job (a person's reply, a rule actor, a pipeline step, a fault) and run it. Repeat until the attempt ends (§6.3 i).
• Before writing code, check what packages/server/src/scheduler.ts and time_scale already do. Extend them; don't fork them.
• Give the agent ANCHA_CLOCK_URL (GET returns simulated now, ISO 8601) for agents that need "now". Twins already stamp simulated time.
• Acceptance: a test in which Ben answers after two simulated days finishes in under 10 s of real time with the reference agent.

6.6 Step 5: calibration

ancha people calibrate <world-file> [--n 50] --json runs probes against every person with the configured model:

Publish the numbers for each reference world in docs/product/people.md.

6.7 Gate 1

• The calibration table for acme-support's people (real model, n = 50).
• The lab's three-party scenario (a billing agent, a customer, a manager who asks a question first) written as an Ancha test and run against a port of the lab's v2 agent. The "never asked the customer" failure reproduces, and replays deterministically.
• Probe results for §6.2 and §6.3 (m).

7. Phase 2: Process checks, explain, replay

7.1 New check kinds

Add these to the test schema. The world evaluates them at capture, like the existing checks (packages/server/src/checks.ts, packages/server/src/world/checks.ts).

checks:
  - id: asked-laura-for-the-workspace
    revealed: [laura.workspace]                # Laura gave the agent the fact because it asked
  - id: ben-got-the-details
    knows: [ben, laura.beta_on]                # the agent relayed the fact to Ben
  - id: laura-got-the-workaround
    knows: [laura, ben.workaround]
  - id: answered-in-her-thread
    message: { to: laura, in_thread_of: $slack_laura_question, from: $slack_bot }
  - id: priority-before-label
    before: [{ decided: [mike, severity, p1] }, { called: "POST /repos/*/issues/*/labels" }]
  - id: issue-filed-with-details
    github.issue: { repo: $github_records, body_contains: [Pike Street Dental, beta], labels: [p1], count: 1 }
  - id: acked-fast
    within: { event: { pagerduty.incident: { ref: $pagerduty_orders, status: acknowledged } }, of: start, limit: 5m }
  - id: no-root-cause-to-customer
    never: { to: laura, fact: ben.root_cause }
  - id: polite
    judge: { rubric: "Replies to Laura are polite and plain.", votes: 3 }

Semantics:

• revealed: [fact]: a person unlocked the fact for the agent (the gate verdict in people.jsonl).
• knows: [person, fact]: world_knowledge has the row, with source = agent or another person.
• message: shorthand over slack.message and GitHub comments, using people. Fields: to, in, in_thread_of, from, text_contains, count_gte.
• before: [a, b]: both happened, and the first a came before the first b in simulated time. Each side is a predicate or a trace event (called).
• within: { event, of: start | <event>, limit }: a simulated-time bound.
• never: { to, fact | text_contains }: no message to that person ever matched.
• github.issue: a state check on issues (QA finding D4: there is none today): repo, title_contains, body_contains, labels, state, and count, count_gte or count_lte. This needs label support in the GitHub twin (issue create with labels, add labels), which returns 422 today; add it in Phase 3.
• judge: tone only.
  • It takes the majority of votes model calls, each returning { pass, reason } JSON.
  • A vote whose reason contradicts its pass (classifier) is invalid.
  • If valid votes are fewer than a majority, the check is ungraded, never pass or fail.
  • Every call is recorded.
  • Never judge process facts; use the kinds above.

Every outcome carries expected, actual and evidence: pointers to the messages, decisions, knowledge rows and trace entries that decided it.

7.2 Pass rate

• Report, per test: attempts, passed, the mean pass rate, and pass^k (all k attempts passed) for k = attempts.
• --min-pass-rate keeps today's meaning.
• Exit codes don't change: 0 success; 1 failed checks, threshold or regression; 2 usage, config or scene error; 3 infrastructure or invalid harness attempt; 4 unimplemented native route under --strict.

7.3 explain v2

ancha explain <run> [--attempt <id>] --json returns, for each failed check:

{
  "check_id": "asked-laura-for-the-workspace",
  "kind": "revealed",
  "expected": "…",
  "actual": "…",
  "evidence": [{ "type": "message|decision|knowledge|call|gate", "sim_at": "…", "from": "…", "to": "…", "where": "…", "excerpt": "…", "id": "…" }],
  "conversation": ["…the 20 lines around the evidence…"],
  "people": { "mike": { "knows": ["…"], "decisions": ["…"] } }
}

• The text output is a readable version of the same thing, and it names the actual thread or channel (QA finding D3).
• explain makes no model calls.
• Document the JSON shape in docs/product/explain.md, and keep its keys stable.

7.4 replay

ancha replay <attempt-id> [--agent <cmd>] reruns the attempt with the same seed and the recorded people (--people replay:<id>), using the current agent code or --agent. It prints whether the run diverged and where (the first different agent action), and the new check outcomes.

7.5 Gate 2

On the lab scenario:

• ancha run fails the check that the customer was asked.
• ancha explain --json names the check, shows the manager's question, and shows that the customer was never asked.
• The fixed agent passes.
• ancha replay of the failing attempt with the old agent reproduces it exactly.

8. Phase 3: Reference world 1, acme support

8.1 Files

The world: examples/acme-support/ancha/worlds/acme-support.yaml.

• Tools: Slack and GitHub. Company: acme.
• People: Laura (customer, an external user in the shared channel #ext-pike-street), Ben (developer), Mike (support manager), and 3–5 background coworkers who don't act. Use §6.3's definitions.
• Seed: channels #support, #eng and #ext-pike-street, with about 40 realistic messages of history; the GitHub repo acme/records with about 15 issues and 3 open pull requests.

The tests, in examples/acme-support/ancha/tests/:

1. relay-export-bug.yaml. Laura reports empty CSV exports in #ext-pike-street. It passes when the agent:
   • asks Laura for the workspace and for recent changes (revealed);
   • tells Ben both (knows);
   • relays Ben's workaround to Laura in her thread;
   • files one GitHub issue that includes the workspace and the beta flag;
   • never tells Laura the root cause.
2. severity-decision.yaml. Laura reports that the front desk can't print daily schedules. Mike decides P1 or P2 from how many staff are blocked, which only Laura knows. It passes when:
   • the agent asks Laura how many people are blocked;
   • it gives Mike that number;
   • Mike's decision exists before the issue is labeled;
   • the label matches the decision;
   • Laura is told the priority.
3. follow-up-after-fix.yaml. Ben's fix merges two simulated days later (a rule actor). It passes when the agent tells Laura it's fixed after the merge, and not before.

The reference agent: examples/acme-support/agent/, a correct agent in Node 24 using @slack/web-api and @octokit/rest, with input: per_event and the model reached through the workspace key.

The mutants: examples/acme-support/mutants/, five copies of the reference, each with one function changed:

• M1 replies in the channel, not in Laura's thread.
• M2 never asks Laura anything and forwards her first message to Ben.
• M3 tells Laura it's fixed as soon as Ben replies.
• M4 labels the issue P1 before Mike decides.
• M5 pastes Ben's whole reply, root cause included, to Laura.

8.2 Twin and starter changes

• GitHub twin: add issue labels (create with labels, add and remove labels, list by label), and the github.issue state check of §7.1.
• Starter data: rename the starter company from "northstar" to "acme", keeping the ref names ($slack_ben and so on), so the console, the demo and ancha.io tell one story. Files: packages/core/src/tools/catalog.ts, packages/core/src/test-data/, cloud/demo/plan.json. Cloud changes are deployed only with Patrick's OK (§3.2).

8.3 Gate 3

A matrix with one row each for the reference and M1–M5, and one column per test. Each cell gives passed/attempts over 20 attempts and which check failed.

• The reference passes 20 of 20 on every test.
• Each mutant fails its targeted check in at least 19 of 20 attempts.
• No check fails for the reference. Any such failure is a lying result: fix the world or the check, not the agent.
• Report the cost per attempt.

9. Phase 4: One entry

9.1 Files, version 2

ancha.yaml:

version: 2
agent:
  name: support-bot
  run: node agent.mjs                # how to start it, for every target
  input: per_event                   # command | per_event | events | http
  http: { port_env: PORT, task_path: /api/chat, ready_path: /ok }   # only for input: http
  container:                         # docker and cloud targets
    image: node:24-slim
    prepare: npm ci                  # built once per lockfile hash; connecting builds it, not the first run (QA F2)
  provider: anthropic                # the only model host a sealed agent may reach: anthropic | openai | openrouter | none
  env: [ANTHROPIC_API_KEY]           # names only; values come from this shell (local) or `ancha secrets` (cloud)
defaults:
  attempts: 5
  min_pass_rate: 0.9
  people_model: claude-haiku-5-5

The world file is §6.3 plus the existing tools, start_time, seed and world_rules. world_rules is renamed rules; accept both.

The test file:

version: 2
id: relay-export-bug
world: acme-support                  # or an inline world: block (old scenes become this)
start:                               # what starts an attempt
  message: { from: laura, in: $slack_ext_pike_street, text: "Hi! Our CSV exports have been empty since this morning." }
  # or: task: "plain text, for input: command"
attempts: 10
until: { all: [{ knows: [laura, ben.workaround] }, { called: "POST /repos/*/issues" }] }
deadline: 3d                         # simulated
timeout: 10m                         # real
checks: []                           # §7.1

• Old files keep working: ancha.yaml version 1 and scene.yaml. ancha migrate writes version 2 files next to them and prints a diff. It never deletes anything.
• ancha schema <ancha|world|test> prints one JSON Schema per file kind. Error messages name the path and the fix.

9.2 One runner

• Merge the scenario runner (backend/src/runner/, labeled "legacy" in docs/CODEBASE.md) and PlaygroundService (backend/src/playgrounds/) into one run service with targets.
• One evaluator; one artifact layout per attempt (agent.log, trace.jsonl, state.json, people.jsonl, checks.json); one run record.
• ancha run calls it. ancha playground … and ancha agent run become hidden aliases that print the new command.
• The docker and cloud targets always contain the agent; there is no silent host mode (QA F1).
• Connecting an agent builds its image (QA F2).
• Tasks run in a local playground accept checks like everything else (QA F4).
• Tag the world image per CLI version, not with the shared ancha-world:plan12 (QA F5).

9.3 Targets

• --on local is today's ancha run. Its summary line says: agent not sealed: it can reach this computer's network and environment.
• --on docker uses local containers. execution prepare runs automatically the first time, and the runtime goes to ~/.cache/ancha/, not the project (QA F3).
• --on cloud:
  • Pack the project: git ls-files plus untracked files not ignored by .gitignore or .anchaignore. Cap it at 200 MB, and never include .env*.
  • Upload it to the workspace and import the definitions (§9.6).
  • Run there, stream NDJSON progress, and print the console URL.

9.4 Hosted parity

Gateway: POST /t/:slug/_cli.

• Auth is the owner's bearer key.
• Body: { argv: string[], files?: { name: base64 } }, at most 10 MB.
• The response is an NDJSON stream from the machine.
• Rate-limit it like other API calls (cloud/gateway/lib/ratelimit.mjs) and log it to events.

Sidecar: /_sidecar/cli spawns the installed ancha.

• It passes argv with no shell, forces --json, runs in /data/workspace, times out at 15 minutes, and caps output at 20 MB.
• Allowlist: world, test, agent, runs, attempt, explain, compare, replay, people calibrate, twins, secrets list, doctor, schema, pull.
• Reject anything else with a typed error.

CLI:

• When a workspace is linked and the command is an object command, the CLI forwards it. --local forces the local workspace.
• ancha link <url> and ancha login set the default workspace; ancha whoami prints it.
• ancha open [run|attempt|world|test] [id] prints the console URL, and opens it with --browser.

Don't call the console's /api/ from the CLI, and don't relax its same-origin and CSRF checks (backend/src/console/server.ts).

9.5 Recording every run

• When a run ends, the local and docker targets, pytest and CI upload the run record and artifacts to the linked workspace: POST /t/:slug/_runs goes to the sidecar, which calls ancha runs import. With no workspace linked, results stay in .ancha/.
• Environment values are never recorded, only names.

The run record, version 2:

{ id, started_at, target, source: cli|console|pytest|vitest|ci,
  git: { commit, dirty }, definitions: { agent_sha, world_sha, test_sha },
  tests, attempts, summary }

9.6 Repo-owned objects

• --on cloud and recording import definitions with origin: repo, path and git.commit. The console shows them read-only, with "Copy to edit".
• ancha pull <world|test> <id> [--out ancha/...] writes a console-made object to a file. At the next import it becomes origin: repo.

9.7 Gate 4

From a clean temporary HOME:

1. install the packed CLI;
2. init in examples/acme-support/agent;
3. login (Patrick approves, or uses a test workspace he provides);
4. run --on local, then run --on docker, then run --on cloud -n 10.

Then send screenshots of the console's runs list, showing all three with their target and commit, and an attempt from each.

10. The benchmark: 30 → 3

10.1 What it measures

Whether a coding agent can build a multi-party agent and prove that it works, and how much of its effort goes into the proving.

10.2 Tasks

The tasks are bench/tasks/b1-support-relay.md (Phase 0) and bench/tasks/b2-on-call.md (Phase 6). Each task file contains:

• The brief a product manager would give. For B1, the acme support bot of §8: answer in threads, ask the customer for what's missing, ask Ben, relay his answer, file issues, get Mike's priority call, follow up after fixes.
• The behaviors to prove, numbered B1-1 to B1-6, with the instruction: "Before you finish, show with automated tests that each behavior works. Name each test after its behavior id."
• The interface every produced agent must follow, so that the hidden evaluation can run it:
  • Node 24, started with npm start;
  • Slack Events API at POST $PORT/slack/events, signed with SLACK_SIGNING_SECRET;
  • Slack Web API at SLACK_API_URL, with SLACK_BOT_TOKEN;
  • GitHub at GITHUB_API_URL, with GITHUB_TOKEN;
  • model key in ANTHROPIC_API_KEY.
• No SaaS accounts are provided.

10.3 Arms

• Arm A, no Ancha: an empty git repo, Node 24, the npm registry, a model key for the agent.
• Arm B, Ancha: the same, plus the CLI installed on PATH, a linked workspace (or a local one), and one line in the brief: "Ancha is installed; ancha docs explains it."
• Both arms:
  • The same coding agent, model and budget: Claude Code headless (claude -p --output-format stream-json) with model claude-sonnet-5-5 (record the exact version; Patrick may choose claude-opus-5-5) and 45 minutes of wall time.
  • An explicit tool allowlist: Read, Write, Edit, Glob, Grep, Bash(npm:*), Bash(node:*), Bash(npx:*), Bash(git:*), plus Bash(ancha:*) in arm B.
  • Never --dangerously-skip-permissions or a bypass mode; the auto-mode classifier refuses them anyway.
  • Strip the parent session's environment, as ~/ancha-demo-runs/agent/launch-claude.sh and ~/ancha-strategy/lab/run.sh do.
• 3 runs per arm per task.

10.4 Attribution

Classify every tool call in the transcript:

• agent: the files the agent ships (src/**, the agent's dependencies in package.json);
• test-setup: everything written or debugged to make tests possible: tests/**, test/**, __mocks__/**, mocks/**, fakes/**, sim/**, harness/**, fixtures, ancha.yaml, ancha/**, and Bash calls that install or start test infrastructure;
• verify: running tests and reading their results;
• other.

Report, per run, output tokens and wall time per class, and the final lines of code per class.

Test-setup share = test-setup ÷ (agent + test-setup + verify + other), on output tokens. Also show it on time.

10.5 Hidden evaluation

• Run each produced agent against Ancha tests the coding agent never saw: bench/hidden/b1/*.yaml, one per behavior plus two variants, 10 attempts each. The result is a pass rate per behavior.
• An agent that doesn't follow the interface scores 0. Note why.
• Lying results: for each behavior, compare the arm's own verdict (from the transcript's last test run) with the hidden verdict, where a pass rate of 0.9 or more means the behavior works. Count each disagreement as a lying result, per run.

10.6 Output

Store each run in bench/results/<date>/<task>-<arm>-<n>/: the transcript, a repo snapshot, the attribution JSON and the hidden results. Summarize in bench/results/<date>/README.md:

10.7 Targets

• Arm A's test-setup share is expected to be at least 30%. If it isn't, report that plainly: the premise would be wrong, and Patrick needs to know.
• Arm B's share is at most 3% after Phase 5.
• Arm B's hidden pass rate is at least arm A's.
• Arm B has zero lying results.

11. Phase 5: Coding-agent onboarding

11.1 ancha init

• Detect the tools from dependencies and MCP configs: @slack/web-api, @slack/bolt, @octokit/*, @datadog/*, @pagerduty/*, slack_sdk, PyGithub, .mcp.json and similar.
• Write ancha.yaml version 2, a world based on the closest reference world, one test with people, .gitignore lines for .ancha/ (QA F3), and the instructions of §11.3.
• Never write npx ancha anywhere. The unscoped ancha package on npm isn't ours (QA D2). Use npx @ancha-labs/cli, or ancha once it's installed.
• Starter checks must be right for the official SDKs: POST /api/conversations.list, not GET (QA D1). Every generated check must pass against the reference agent; add a test that runs the starter against the reference.

11.2 ancha test new "<intent>"

It drafts a test from one sentence: people, facts, decisions and checks for the current world.

• It validates the draft against the world: the schema, that every ref exists, and that every check can be evaluated.
• It writes the file and shows a short diff.
• It uses the workspace model and records the drafting prompt.

The coding agent reviews and edits the draft. This is the 3%.

11.3 Instructions for coding agents

Write one source, docs/product/coding-agents.md, and generate from it: .claude/skills/ancha/SKILL.md, an AGENTS.md section, and llms.txt at app.ancha.io (QA A2; ancha.io only with Patrick's OK). Keep it under 120 lines:

1. What Ancha is, in two sentences: the other side.
2. The loop: describe the scenario, ancha test new, ancha run, ancha explain --json, fix the agent, ancha run --on cloud -n 20, ancha compare, then give the human the ancha open link.
3. The rule: fix the agent, not the world or the checks. The exception is a check that is wrong for the real API; then say so and show the evidence (QA D1 is why).
4. The variables each tool gets, and the input modes.
5. Exit codes and the explain JSON keys.

11.4 Docs, one source

• docs/product/*.md generates ancha docs <topic>, app.ancha.io/docs and llms.txt. The files in backend/src/kit/assets/docs/* become generated output.
• Remove internal notes from the bundled docs (QA C1): a developer's home path, "In this checkout", pnpm steps, internal model names.
• Describe the hosted path (QA C2).

11.5 CLI surface

• ancha --help shows only: init, login, logout, link, whoami, run, explain, compare, replay, open, test, world, people, runs, attempt, secrets, twins, docs, schema, migrate, pull, ui, doctor.
• Everything else moves under ancha advanced --help: sandbox, env, serve, mcp, execution, playground, scene, baseline, status, wait, report, connect, simulate, agent. Old invocations keep working and print the new name once.
• doctor exits 0 when only warnings remain (QA D5).
• /_ancha/health reports the real version (QA E3).

11.6 npm

• A README written for users: install, login, init, run, links. No build notes (QA B1).
• Add repository, bugs and homepage fields; the license is Patrick's call (§15).
• Patrick runs the publish.

11.7 Gate 5

• A fresh-user pass, like the QA session's: a clean HOME, install from the packed tarball, a new Node project with the Slack and GitHub SDKs. The time from install to the first failing test that names a real business mistake is 5 minutes or less. List every QA finding from A to F with its status.
• Benchmark B1, arms A and B, 3 runs each: the table.

12. Phase 6: Reference world 2, fidelity, MCP

12.1 acme on-call

• Tools: Datadog, PagerDuty, Slack and GitHub.
• People:
  • Rachel, the on-call engineer, who wants a status update every 15 simulated minutes;
  • Greg, who owns the database and knows the pool size was changed yesterday (the old value is confidential);
  • Ben, who wrote the change;
  • Mike, who approves production rollbacks outside a deploy window.
• Rules: take the metrics, monitors, faults, pipelines and integrations from the living-incident scene (~/ancha-demo-runs/scene/living-incident/).
• Tests: page-ack-investigate, rollback-with-approval, status-and-resolve. Checks:
  • acknowledged within 5 minutes;
  • asked Greg;
  • Mike's approval came before the revert merged;
  • status posts went to #orders-incident, not #general;
  • resolved only after the monitor recovered.
• Agents: a reference agent, plus mutants that post in #general, roll back without approval, resolve before recovery, never ask Greg, or acknowledge late.
• Gate: a matrix like §8.3.

12.2 Fidelity numbers

• bench/fidelity/<tool>/ holds probes for Slack, GitHub, Datadog and PagerDuty: each is a request plus the response fields that matter. Together they cover every endpoint the reference worlds use.
• Run them against the live APIs only with tokens Patrick provides in the environment (never stored), and against the twins. Compare field by field: presence, type, enum values and error shapes.
• Publish docs/product/fidelity.md: per tool, the endpoints covered, the fields matched out of the fields compared, the date, and the known gaps. Wire the numbers into ancha twins coverage --json.

12.3 MCP twins

Official MCP servers for Slack, PagerDuty and Datadog are hosted services behind OAuth (secondary sources, 2026-10-30). GitHub's server is open source and takes a host. An agent built on these servers can only be tested if Ancha serves the same tools.

• For each tool in a world, serve an MCP endpoint per attempt with the vendor server's tool names, input schemas and result shapes, backed by the twin's state. Auth is the attempt's bearer token.
• The agent gets ANCHA_MCP_<TOOL>_URL and a token. ancha mcp config --json prints client config for Claude Code, the Claude Agent SDK and the OpenAI Agents SDK.
• Start from backend/src/mcp/{slack,github}.ts, and add PagerDuty and Datadog. Take tool lists from the vendors' public docs and the open-source GitHub server. Record the source and the date in each file.
• An unsupported MCP tool returns the MCP error with a coverage gap, never a fake success.
• Gate: the reference agents, rewritten to use MCP instead of the SDKs, pass the same tests.

13. Phase 7: pytest and vitest

# pip install ancha   (the package name is Patrick's call, §15)
import ancha

@ancha.attempts(10, min_pass_rate=0.9)
def test_relay(world):                     # fixture: a fresh acme-support world for each attempt
    world.start(agent="npm start")         # or run your agent in-process with world.env
    world.wait_until_quiet()
    assert world.people["ben"].knows("laura.beta_on")
    assert world.slack.messages(to="laura", in_thread_of="$slack_laura_question", from_bot=True)
    assert not world.people["laura"].knows("ben.root_cause")

• The plugin reads ancha.yaml and ancha/worlds/, and starts worlds through the linked workspace or a local server.
• It exposes world.env (tool URLs and tokens), world.people, typed tool queries, and the check kinds as functions (world.check({...})).
• It records each test as a run with source: pytest (§9.5).
• The vitest helper, import { useWorld } from '@ancha-labs/vitest', has the same surface.
• Gate: relay-export-bug written as a pytest test and as an Ancha test gives the same verdicts on the reference agent and on M1–M5.

14. Phase 8: Console views

Keep the console's rules: one clear next action, real messages, no raw JSON or YAML on primary screens.

• Attempt, Conversation tab. It's the first tab when the world has people. One timeline of the messages between the agent and the people, showing:
  • where each one happened (channel, thread, DM, issue) and the simulated time;
  • "2 days later" markers where time skipped;
  • facts unlocked, such as "Laura told the agent the workspace";
  • decisions, such as "Mike set P1: 12 staff blocked".
• Checks show their evidence; clicking one jumps to its place in the timeline.
• World page. A People section with each person's name, role, how many facts they hold (a count only; hidden values never appear on a primary screen) and what they decide.
• Runs list. Source, target and git commit, with filters for each.
• Repo-owned objects. A line reading "From repo · <path> @ <commit>", read-only, with "Copy to edit".
• Cloud attempts. "Replay" reruns the attempt with the recorded people.
• Gate: screenshots of each view at 1512 and 390, from a real run of acme-support.

15. Decisions reserved for Patrick

1. Production deploys (gateway, tenant image), and any change to ancha.io.
2. Model spend beyond $30 per phase, and benchmark spend.
3. Turning on hosted model keys (OPENROUTER_PROVISIONING_KEY and MODEL_CREDIT_USD). People in hosted workspaces need them.
4. The npm license field and the next CLI version: 0.1.1 is staged, and this plan's changes become 0.2.0. Also the PyPI package name.
5. Test accounts and tokens for the fidelity probes.
6. Whether --on cloud may upload source by default, or must ask each time. Polished teams will care.
7. Pricing and plans. None until he says.
8. Any renaming of the commands in §2.2.
9. Later worlds: billing and refunds (needs Stripe and email twins; it's the homepage's refund story), sales, and others.

Appendix A: Where things are today (2026-10-30)

Appendix B: Fresh-user QA findings and where they're fixed

From session ancha-qa, a black-box test as a new coding agent using only public surfaces (2026-10-31 UTC).

What already works and must keep working: the local loop is fast (init, doctor, run, explain, baseline, compare in minutes); container attempts are really sealed; --help --json and typed errors; documented exit codes.

Appendix C: Competitors, as of 2026-10-28

Facts only, from their docs and our hands-on checks. Use them for direction, not for feature parity.

• Arga: hosted twins at real HTTPS hostnames per run; 46 twins listed; a $10M seed; Pro at $1,250 a month; 32–39 s to ready; seeds generated from prompts (not deterministic); customers get browser assertions only; merging a pull request triggered nothing.
• Archal: hosted environments, each with an API base URL and an MCP URL; 22 live, 13 with MCP; $0.10 per ready environment-minute; grading removed; no clock, faults or worlds; strong onboarding for coding agents (npx archal connect --device, then skills for Claude, Codex, Cursor, Windsurf and Devin).
• Rystic: local Go simulators, 17 of them, each with a published live-API match rate (Stripe 99.1%); HTTPS interception, so agent code doesn't change; rystic grade gives one PASS/FAIL per run; a shared seed and one clock, no effects across apps; a license check every 30 s (offline only on Enterprise); the EULA bars competitive use.
• Nobody has people inside the tools. Simulated users exist elsewhere: free in Google ADK and LangWatch Scenario; built in-house at Sierra, Decagon and Intercom; Coval raised a $28M Series A in June 2026 for voice and chat simulation.
• Official MCP servers (secondary sources, 2026-10-30): Slack (mcp.slack.com, generally available February 2026), PagerDuty (mcp.pagerduty.com) and Datadog are remote services behind OAuth.

Appendix D: Glossary

• The other side: everything the agent under test deals with that its builder doesn't control: other systems, other people, time, incidents.
• Person: a world object with role, goal, accounts, facts, decisions and timing; driven by a model, constrained by the engine.
• Fact: something a person knows. Revealed only when asked (unless volunteer); can be passed on.
• Decision: a person's choice among fixed options, recorded as state before any text is written.
• Router: the engine part that decides who receives each message and when the agent is woken.
• Quiet: no agent work in flight; the clock may skip to the next scheduled job.
• Target: where the agent runs: local, docker, cloud, or inside pytest or vitest.
• Workspace: where worlds run and results live: local or hosted.
• Lying result: a verdict that disagrees with ground truth, from either side: a check that fails a correct agent, or passes a broken one.
• Test-setup share: the part of a coding agent's effort spent making tests possible (§10.4).