Operational Safety

Runtime monitoring of conversation content, caller emotion, and agent confidence with graduated escalation to human operators.

Operational safety governs what happens during a live conversation. While a call is in progress, the platform continuously evaluates the agent's behavior, the caller's state, and the agent's confidence in its own understanding. When something falls outside expected parameters, the system adjusts the agent's autonomy or escalates to a human operator.

What Gets Tracked

During every conversation, the platform monitors several categories of signals:

Conversation Content

The monitoring system evaluates what is being said against semantic rules (monitor concepts). These rules detect when the conversation touches topics that require special handling: clinical advice, medication dosing, mental health crisis indicators, insurance disputes, or any domain-specific concern configured for the workspace.

Caller Emotional State

The emotion detection system provides a continuous read on the caller's emotional state. Sustained distress, escalating frustration, or signs of crisis are tracked as part of the safety picture. A caller who is becoming increasingly agitated triggers a different response than one who is calmly asking routine questions.

Agent Confidence

The agent maintains a measure of how well it understands the current situation. When confidence is high, the agent operates with full autonomy within its context graph. When confidence drops - because the caller's request is ambiguous, the required information is not available, or the conversation has moved into unfamiliar territory - the agent's behavior narrows.

This is not a binary on/off switch. Confidence is a spectrum, and the agent's autonomy adjusts proportionally:

Confidence Level	Agent Behavior
High	Full autonomy. Agent navigates the context graph, executes actions, and responds without restriction.
Moderate	Agent continues operating but hedges its responses, seeks confirmation from the caller, and avoids committing to actions it is unsure about.
Low	Agent acknowledges uncertainty to the caller and escalates to an operator if one is available. If no operator is available, the agent offers to have someone call back.

Escalation Signals

Escalation signals are not evaluated in isolation. The platform considers them together - a single borderline signal may not trigger escalation, but two moderate signals occurring together may.

Multiple signal types can trigger escalation to an operator:

• A monitor concept fires (the conversation has entered territory that requires human oversight)

• The caller explicitly asks to speak with a person

• Agent confidence drops below the workspace's configured threshold

• The emotion detection system identifies sustained caller distress

These signals are not evaluated in isolation. The platform considers them together. A single borderline signal may not trigger escalation, but two moderate signals occurring together may.

How Escalation Works

When the platform determines that a call needs human involvement, the escalation follows a defined sequence:

1. Notification - The operator on duty receives a notification with the call context: caller identity, conversation summary, reason for escalation, and the relevant signals.

2. Join - The operator joins the live conference call in listen mode. They hear the conversation and review the transcript.

3. Decision - The operator decides whether to take over the call or let the agent continue with monitoring.

4. Takeover (if needed) - The operator switches to takeover mode. The agent's audio is suppressed and the operator speaks directly with the caller.

5. Return - When the operator resolves the situation, they switch back to listen mode or leave the call. The agent resumes with full context.

The caller experiences this as a single continuous call. There is no transfer, no hold, no disruption.

Risk Scoring

The voice agent computes a composite risk score on every turn from three weighted signals: caller emotion (40%), conversation loop detection (30%), and call duration relative to expected length (30%). The score maps to four levels - normal, monitor, alert, and escalate - with configurable thresholds per workspace. Individual context graph states can override the default thresholds when certain conversation phases carry higher clinical risk. See Risk Scoring and Silence Management for the full breakdown.

Safety Triage

In addition to the composite risk score, the voice agent runs regulation-specific triage on every turn. Pre-built templates detect patterns associated with suicide risk (Joint Commission NPSG 15), domestic violence (VAWA), and adverse drug reactions (FDA MedWatch). Each template includes triage hints - specific linguistic and behavioral patterns the LLM should watch for beyond direct statements. See Safety Triage for the full framework.

Confidence-Gated Responses

The agent does not just track confidence internally. It uses confidence to gate what it is willing to do:

• Scheduling actions require moderate confidence or higher. The agent will not book or cancel an appointment if it is unsure it understood the request correctly.

• Clinical data extraction writes events at voice confidence (0.5) regardless of agent confidence. The confidence gates in the connector runner handle verification before anything reaches the EHR.

• Information delivery adjusts based on confidence. When the agent is highly confident, it states facts directly. When confidence is lower, it qualifies its statements ("Based on what I see in your record..." rather than "Your appointment is...").

This gating ensures that the agent's willingness to act matches its understanding of the situation. An agent that is unsure should sound unsure, and should limit its actions accordingly.