> For the complete documentation index, see [llms.txt](https://docs.amigo.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.amigo.ai/operations-and-safety/runtime-safety.md). # Runtime Safety Runtime safety is a layered system. Authored conversation policy guides the agent, structural controls constrain supported execution paths, server-side authorization limits writes, and operators can intervene through the unified Runs surface. The current architecture does not depend on a separate semantic monitoring or regulatory-triage service. Safety-critical behavior must be expressed in the agent's context graph, tool permissions, escalation instructions, and operating procedures, then verified against representative scenarios. ## Control Map | Control | Scope | Current Behavior | | --------------------------------------- | -------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Context graph policy** | Voice and text | States define objectives, available tools, exit conditions, guardrails, and boundary constraints that shape agent behavior. | | **Graph navigation invariants** | Platform-managed context graph runtime | Navigation resolves to an engageable state, detects loops, and caps per-turn transitions in code. | | **Cross-runtime invariant observation** | Voice | The platform records shadow verdicts for engageable-state, loop, and iteration checks. These verdicts do not change the live call. | | **Response guardrail observation** | Voice | Candidate responses are compared with the active state's guardrails and boundary constraints in shadow mode. Verdicts are diagnostic only. | | **Integration approval policy** | Text conversations | A top-level integration tool call can pause for an authorized approve or reject decision when its integration policy gates the operation. | | **Write scope** | Supported agent write tools | A server-side write scope binds model-originated writes to an allowed workspace and entity; each registered tool constrains its resource shape and source-appropriate confidence. | | **In-flight write deduplication** | Supported voice runtimes | An identical write-tool call is short-circuited while the first invocation is still running. A later invocation can proceed after the first finishes. | | **Outbound policy** | Connector delivery | Source, confidence, schema, dependency, and destination rules determine whether an event is eligible for unattended delivery. | | **Operator control** | Eligible live runs | Authorized operators can listen, take over, send guidance, author a text turn, and hand control back, depending on channel. | This map distinguishes controls that actively enforce behavior from controls that only produce evidence. That distinction matters when setting clinical and operational expectations. ## Safety in the Agent Policy The agent evaluates the conversation within its normal reasoning loop. There is no independent subsystem that can compensate for missing escalation instructions or incomplete boundaries. For each safety-sensitive state, define: * **Objective** - What the agent is trying to accomplish in that state. * **Allowed tools** - The smallest tool set needed for the task. * **Boundary constraints** - What the agent must not claim, recommend, or attempt. * **Escalation conditions** - Observable situations that require a human or a configured transfer target. * **Safe fallback** - What the agent should say and do if the required person, tool, or destination is unavailable. * **Exit conditions** - The evidence needed before the graph advances. On the Platform API, reusable policy that spans states should be expressed through shared context graph structure, state guidance, transitions, eligible tools, and tested authoring conventions. Classic dynamic behavior sets are a separate Classic API capability and are not a Platform runtime control. {% hint style="warning" %} Safety instructions are model-facing policy. They are not a substitute for server-side authorization, clinical governance, or a staffed escalation process. {% endhint %} ## Structural Navigation Controls The platform-managed context graph navigator enforces three structural properties during a turn: 1. **Engageable landing** - Routing states are resolved until the conversation reaches a state that can handle the next user interaction. 2. **Loop handling** - Repeated navigation is detected so the engine does not traverse the graph indefinitely. 3. **Transition cap** - A bounded number of state changes can occur within one turn. If navigation cannot resolve safely, this runtime stays on the last engageable state rather than presenting a routing-only state to the caller. The voice layer also observes these properties across supported runtimes. That observer is shadow-only: it records diagnostic verdicts but does not interrupt, reroute, or end a call. Runtimes that navigate independently therefore do not inherit the platform-managed navigator's enforcement merely because they produce the same telemetry. ## Guardrail Evaluation Status State guardrails and boundary constraints are included in the agent's active instructions. Separately, the platform runs a lightweight response-text evaluator on voice calls and records whether a candidate response appears to overlap a configured rule. {% hint style="warning" %} The response evaluator is shadow-only. A recorded `block` verdict does not block or rewrite the response, and evaluator failure does not affect the call. Treat these verdicts as calibration and regression evidence, not as an enforced clinical safety boundary. {% endhint %} Because shadow evaluation observes text rather than clinical truth, teams should review false positives and false negatives before using its output in an operational process. ## Write Safety ### Integration Tool Approval Workspace integrations can set an approval policy of `none`, `writes`, or `all`. On supported text conversation paths, a gated top-level integration call is parked before execution until an authorized reviewer approves or rejects it. The default is `none`. This mechanism is conversation-scoped and fail-closed. It does not currently gate integration tools called from companion skills, and it does not apply to voice execution. It is also separate from external write proposals: integration approval pauses a conversation tool call, while an external proposal stages a connector payload for later review and delivery. ### Agent Write Scope Supported agent write tools receive a server-constructed write scope that restricts which workspace and entity the operation can target. The registered tool contract separately constrains the resource shape and assigns or caps the confidence available to model-originated data. These checks protect the persistence boundary even if the model supplies an unexpected identifier. System-originated connector and projection work follows separate trusted-service authorization and is not evidence that an agent can bypass its own scope. ### Duplicate Calls Supported voice runtimes track identical write-tool calls that are already in progress. If the model repeats the same write before the first completes, the duplicate is rejected as already running. This prevents concurrent duplicates; it is not a permanent idempotency guarantee for later calls. Operations that need durable idempotency should still use the operation's documented idempotency key or destination-specific protection. ### External Delivery Recording an observation in the world model does not confirm an external mutation. Unattended connector delivery evaluates the event against its source, confidence, schema, dependencies, and configured destination. The standard automated outbound path is at-most-once and has no reconciliation loop. A transient failure can remain a permanent failed delivery unless the customer verifies and retries it through an appropriate workflow. Review-gated external write proposals are a separate private-preview capability. Where enabled, a source-eligible connector event for a review-configured destination is staged at any confidence, and an authorized reviewer can approve or reject that specific payload and target. Approved FHIR-sink proposals can be retried under their idempotent delivery policy; other sink types receive at most one automatic vendor call and then require investigation or manual reconciliation after an ambiguous failure. This flow does not restore the retired general review queue and does not make every world-model observation human-reviewed. ## Human Intervention The unified Runs surface gives operators one place to monitor eligible live work: * **Voice** - Join in listen mode, explicitly take over, send guidance, switch mode, and hand the call back. * **Text channels** - Pause an eligible run, author the next turn, and hand control back. * **History** - Inspect the transcript, run metadata, available analysis, and recorded operator actions. Operator actions require the Operator update permission, available to operator, admin, and owner roles. Supported action routes bind the requested operator identity to the authenticated caller and emit operator-attributed events for takeover-related actions. An authored escalation policy must also define what happens when no operator is available. Appropriate fallbacks can include a configured transfer target, callback workflow, emergency instruction, or a bounded refusal. Do not assume the platform automatically pages a clinician or creates a post-call review item. ## Validation Before Release Simulations exercise the current workspace configuration through the real reasoning and tool path. Simulation-originated world-model writes are tagged so production EHR delivery and production analytics can exclude them, but simulations do not create an isolated configuration branch and do not replace every external dependency with a stub. Use dedicated test workspaces, test entities, or safe integration environments when a scenario could reach a real external system. Separate broad text-tier validation from real-audio tests that measure transcription, speech rendering, turn timing, and media attachment. A practical release review includes: 1. Run targeted cases for each safety boundary and escalation path. 2. Run regression suites against the configuration intended for release. 3. Inspect normalized checks, transcripts, tool calls, and failure reasons. 4. Review shadow guardrail and control-invariant evidence where available. 5. Confirm operator staffing and fallback procedures. 6. Deliberately update the pinned `release` version set when the evidence is acceptable. Simulation results and drift analysis do not automatically block version-set promotion or roll back a release. Promotion and rollback are deliberate operator actions. ## Production Review After release, use conversation evaluations, run history, connector outcomes, and operational metrics to find changes in behavior. Compare defined cohorts and stable time windows rather than treating a single aggregate score as proof of safety. When a regression appears: 1. Identify affected channels, services, versions, and cohorts. 2. Add or update a reproducible simulation case. 3. Verify whether the issue is agent behavior, tool behavior, media quality, or destination failure. 4. Correct the smallest responsible configuration or integration boundary. 5. Re-run the affected suite before moving the release pin. ## What Runtime Safety Does Not Guarantee * **Perfect outcomes** - Model behavior and external systems remain fallible. * **Automatic clinical triage** - Organizations must author, test, and staff their own clinical escalation protocols. * **Enforced shadow verdicts** - Guardrail and cross-runtime invariant observers currently record evidence only. * **Universal human review** - Only supported top-level text integration calls and enabled private-preview external write proposals use their respective approval paths. * **Automatic deployment rollback** - Teams decide when to move a version-set pin. * **Automatic outbound reconciliation** - Standard unattended connector delivery is at-most-once. {% hint style="info" %} Triage and safety evaluation support operational decisions; they do not replace clinical assessment. Organizations remain responsible for clinical protocols, escalation staffing, destination verification, and regulatory review. {% endhint %} ## Related Documentation * [Context Graphs](/agent/context-graphs.md) * [Operators](/operations-and-safety/operators.md) * [Testing and Evaluation](/testing/testing.md) * [Connectors and EHR](/data/connectors-and-ehr.md) * [Compliance](/operations-and-safety/compliance.md) * [Safety and monitoring APIs](https://docs.amigo.ai/developer-guide/platform-api/safety) --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.amigo.ai/operations-and-safety/runtime-safety.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.