> For the complete documentation index, see [llms.txt](https://docs.amigo.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.amigo.ai/developer-guide/platform-api/safety/simulation-coverage.md). # Simulation Coverage The simulation coverage API provides branch-and-bound exploration of context graph state space. It replaces random sampling with targeted exploration that steers conversations toward untested states, tools, and transitions. {% hint style="info" %} **Conceptual overview.** For background on how coverage testing works, see the [Simulation Coverage](https://docs.amigo.ai/testing/simulations#simulation-coverage) conceptual documentation. {% endhint %} ## Core Model ```mermaid %%{init: {"theme": "base", "themeVariables": {"primaryColor": "#D4E2E7", "primaryTextColor": "#100F0F", "primaryBorderColor": "#083241", "lineColor": "#575452", "textColor": "#100F0F", "clusterBkg": "#F1EAE7", "clusterBorder": "#D7D2D0"}}}%% flowchart TB subgraph Run["Coverage Run (simulation-tagged writes)"] S1["Session 1"] --> T1a["Turn 1"] --> T1b["Turn 2"] --> T1c["Turn 3"] T1c -->|Fork| S2["Session 2\n(child)"] T1c -->|Fork| S3["Session 3\n(child)"] S2 --> T2a["Turn 4"] S3 --> T3a["Turn 4"] end Run --> Graph["Knowledge Graph"] subgraph Graph_Detail["Graph Layers"] OBS["Observed: per-state\nsession counts + pass rates"] GHOST["Ghost nodes: states\nnever reached"] end Graph --> Graph_Detail ``` Coverage is organized into three levels: | Resource | Description | | -------------------- | -------------------------------------------------------------------------------------------- | | **Coverage Run** | A campaign that contains simulation sessions and tags their writes as simulation data. | | **Coverage Session** | A conversation path within a run. Sessions can be forked from other sessions at any turn. | | **Coverage Turn** | A single agent turn within a session, recording context graph state, tool calls, and scores. | Sessions form a tree structure through the fork mechanism. Each session tracks its `parent_session_id` and `fork_turn_index`, so the full exploration tree is reconstructable from the session list. ## Endpoints All endpoints are scoped to a workspace and require a workspace API key. ### Create Coverage Run {% openapi src="" path="/v1/{workspace\_id}/simulations/runs" method="post" %} {% endopenapi %} Returns the created run with ID, status (`running`), tags, and optional branch label metadata. The label does not create a separate database branch. ### Create Coverage Session Creates a new conversation session within a run. The session starts with a fresh agent state. {% openapi src="" path="/v1/{workspace\_id}/simulations/runs/{run\_id}/sessions" method="post" %} {% endopenapi %} ### Step a Session Sends a simulated user message and records the agent's response as a turn. {% openapi src="" path="/v1/{workspace\_id}/simulations/sessions/{session\_id}/step" method="post" %} {% endopenapi %} Returns the turn observation including agent response, context graph state, tool calls, and turn index. ### Fork a Session Atomically clones the parent session state N times, steps each clone with a different simulated message, and returns all observations. This is the core branch-and-bound primitive. {% openapi src="" path="/v1/{workspace\_id}/simulations/sessions/{session\_id}/fork" method="post" %} {% endopenapi %} Each child session inherits the parent's full conversation history up to the fork point. The parent session remains unchanged. ### Promote a Session Interactive playground sessions are created without a coverage run, so run-scoped operations like [fork](#fork-a-session) and [score](#score-a-session) are unavailable on them. Promotion binds a run-less session to a newly created coverage run, which unblocks fork and score. This lets developers iterate in the playground and then promote interesting sessions for deeper analysis without re-running the conversation. {% openapi src="" path="/v1/{workspace\_id}/simulations/sessions/{session\_id}/promote" method="post" %} {% endopenapi %} The response returns the `session_id` (unchanged), the `run_id` of the coverage run the session is now bound to, and `already_bound`. Promotion is idempotent: a session that was already run-scoped returns `already_bound: true` with its existing `run_id` unchanged. ### Score a Session Evaluates the session against configured metrics. Scores are attributed per-session, not per-state-visit. {% openapi src="" path="/v1/{workspace\_id}/simulations/sessions/{session\_id}/score" method="post" %} {% endopenapi %} Returns a confirmation status (`{ "status": "scored" }`). The supplied score and rationale are persisted; this endpoint does not return computed metric scores. Aggregated metric and coverage scores are read back through the coverage graph, the session list, or the benchmark results endpoint. ### Get Coverage Graph Returns the knowledge graph for the run. The graph has two layers: | Layer | Description | | -------------------- | --------------------------------------------------------------------------------------------------------------------------- | | **Observed turns** | Aggregated per context graph state: session count, pass rate, total turns | | **Topology overlay** | Ghost nodes for context graph states that exist in the state machine but were never reached. These represent coverage gaps. | {% openapi src="" path="/v1/{workspace\_id}/simulations/services/{service\_id}/graph" method="get" %} {% endopenapi %} ### Complete a Coverage Run Marks the run as finished. Sessions and turns remain available for read access. {% openapi src="" path="/v1/{workspace\_id}/simulations/runs/{run\_id}/complete" method="post" %} {% endopenapi %} ### Get Coverage Run Returns the full run payload, including the stored `bridge_request` (original request body) and `scenarios` (generated persona/scenario list) for runs created through the [Simulation Bridge](#simulation-bridge). The list runs endpoint omits these fields to keep responses compact. The response also includes a `checks` array alongside the raw `eval_results` - a normalized, read-only projection that flattens each metric and assertion verdict into one uniform shape. Per-case assertion detail responses carry the same `checks` array. Use it for consistent rendering; the raw fields are unchanged. {% openapi src="" path="/v1/{workspace\_id}/simulations/runs/{run\_id}" method="get" %} {% endopenapi %} ### Get Simulation Performance The performance operation aggregates recent graded case and suite runs into one overview. Use `service_id` to restrict the result and `limit` to bound the recent runs analyzed. When `metrics[].checks` is present, use it for a consistent display across metric and assertion verdicts. It normalizes the source, key, label, verdict, expected and actual values, optional score, rationale, references, and run/session identifiers while preserving `results` for callers that need the raw evaluation records. Do not require either metric-detail field unless the OpenAPI schema for your assigned deployment exposes it; deployments still on the compact response return only `metric_key`, `label`, and `per_run`. {% openapi src="" path="/v1/{workspace\_id}/simulations/performance" method="get" %} {% endopenapi %} ## Simulation Bridge The simulation bridge turns a natural language test objective into a set of autonomous test scenarios and runs them end-to-end against a service. {% openapi src="" path="/v1/{workspace\_id}/simulations/bridge" method="post" %} {% endopenapi %} The bridge can infer a target spec when none is supplied, steer scenario generation toward coverage gaps, fork conversations when enabled, and run generation asynchronously. When asynchronous generation is enabled, poll [Get Coverage Run](#get-coverage-run) for progress. Returns the created coverage run with generated scenarios. Each scenario includes a persona (name, background, temperament) and conversation strategy (initial message, instructions). When `async_generation` is true, the response returns `status: running` with an empty `scenarios` array. Use the returned `run_id` with [Get Coverage Run](#get-coverage-run) to read generated scenarios and final status as they become available. When automatic target-spec inference is enabled, the response includes the inferred spec and its rationale. The bridge response itself carries no execution summary. Read `total_sessions` and per-session results back through [Get Coverage Run](#get-coverage-run) once execution finishes. Each session is tagged with metadata for joining sessions back to their source scenarios: `scenario_index:N` identifies which generated scenario produced the session, and `temperament:X` records the persona temperament used. The bridge creates a tracked coverage run, so results are visible through the existing coverage graph, session list, and scoring endpoints. This is the REST API equivalent of `forge platform sim bridge` - same scenario generation and autonomous execution, accessible without the CLI. ## Saved Case Benchmarks Saved-case benchmarks run a tag-selected batch of simulation cases and queue each selected case as its own bridge run. The benchmark response returns immediately with a stable `batch_id`, the selected case IDs, started run IDs, skipped cases, failed-to-start cases, and an aggregate start summary. {% openapi src="" path="/v1/{workspace\_id}/simulations/benchmarks/run" method="post" %} {% endopenapi %} ### Results Use the returned `run_ids` to aggregate completion status, scores, pass counts, metric availability, per-run summaries, and capability breakdowns. {% openapi src="" path="/v1/{workspace\_id}/simulations/benchmarks/results" method="post" %} {% endopenapi %} ## Bridge Plan Generates a target spec from a natural language objective without running any simulations. Use this to preview and edit the inferred spec before passing it to the bridge endpoint. {% openapi src="" path="/v1/{workspace\_id}/simulations/bridge/plan" method="post" %} {% endopenapi %} The response includes the inferred `target_spec` (same structure as the bridge request `target_spec` field), a `rationale` explaining why each state and tool was included, and `available_states` and `available_tools` listing every state and tool in the service's context graph for UI autocomplete when editing the spec. ## Write Isolation Coverage sessions read the workspace's current configuration and data. Available tools execute through the normal text-agent path rather than through stubs, and tool writes are tagged with `source="simulation"`. Platform workflows that honor simulation-source isolation exclude those writes from production processing, and restricted Surface tools remain unavailable. Source tagging is not a separate database branch, a transaction rollback, or a universal side-effect sandbox. An enabled tool or integration can still contact an external system. Run simulations in a dedicated test workspace with test identities and non-production connector credentials when a real side effect would be unsafe. This is Tier-1 text simulation: the simulated caller sends text through the production reasoning and tool path. It does not test telephony, media attachment, speech recognition, text-to-speech, or other audio behavior. Use a real-audio test workflow for those concerns. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.amigo.ai/developer-guide/platform-api/safety/simulation-coverage.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.