Reasoning Engine

The modality-independent reasoning core that processes signals and emits effects, powering voice, text, simulation, and API agent interactions through a unified pipeline.

The reasoning engine is the core intelligence layer of the Amigo platform. It gives supported voice, text, simulation, and API interactions a shared contract for context-graph navigation, tool execution, and response generation. Channel adapters still own transport-specific behavior such as audio streaming, message delivery, and connection lifecycle.

Why a Unified Engine Matters

Early voice AI systems tightly couple reasoning logic with audio transport. The agent's decision-making is interleaved with speech-to-text timing, filler audio generation, and WebSocket management. This coupling means every new channel (SMS, simulation, API webhooks) must reimplement the reasoning loop from scratch, and bugs fixed in one channel don't propagate to others.

Amigo separates these concerns. Modality adapters convert channel input into typed signals and translate engine output into channel behavior, keeping audio and messaging transport out of the core reasoning contract.

Cut / Navigate / Engage

Amigo uses three operations to coordinate the voice timeline and signal-driven text flow:

1. Cut - Decide whether an incoming signal creates a boundary in the current interaction phase.
2. Navigate - Select the next context-graph state or channel-level unit of work from the current session state.
3. Engage - Carry out that selection by generating a response, executing tools, scheduling a deadline, or delivering channel output.

The operations appear at more than one layer:

Scale	Cut	Navigate	Engage
Conversation	A user turn or qualifying event prompts evaluation	Select a path through the context graph	Generate a response and execute tools
Voice timeline	A timing or session signal creates a boundary	Select breath, filler, hold, response, or listening behavior	Schedule or play the selected audio behavior
Text flow	A queued message or event becomes the next item	Run the shared reasoning path	Deliver or persist the resulting effects

This coordination keeps fillers, silence handling, empathy pauses, tool progress narration, and barge-in recovery on one voice timeline. The voice timeline describes how it operates within each turn.

Signals and Effects

The engine communicates through two primitives.

Signals represent something that happened. Supported adapters normalize relevant input into typed signals before it enters the shared flow:

Signal	What It Represents
Utterance	The caller or user said something (text, from any source)
Emotion	An emotional state update from acoustic or transcript analysis when enabled
Tool result	A tool execution completed with a result
Silence	The caller has been silent beyond the configured threshold
Barge-in	The caller interrupted the agent mid-speech
External event	An injected event from an operator, surface submission, or external system
System	A system-level event such as a timeout, connection change, or error condition

Effects represent something the engine wants to happen. The modality adapter decides how to execute each one:

Effect	Voice	SMS	Simulation
Respond	Stream the generated response through the voice pipeline	Send as an SMS message	Capture as simulation output
Filler	Play filler audio ("Let me check on that...")	No-op	No-op
Pause	Hold deliberate silence	No caller-facing output	Capture the effect when applicable
Tool call	Execute tool, feed result back as signal	Same	Same real tool path, with simulation-tagged writes
Terminate	Hang up after final speech	End session	Return final state
Transition	Records a state change; no caller-facing output	Same	Capture in trace log
Observe	Emits an analytics event; no caller-facing output	Same	Capture in trace log

The Pipeline

Each signal flows through three stages.

Perceive. The modality adapter converts raw input into typed signals. A voice adapter produces utterance signals from speech-to-text and emotion signals from prosody analysis. An SMS adapter produces utterance signals from message text. A simulation adapter injects both from test parameters.

Reason. The engine's core loop implements cut/navigate/engage at the conversation level:

1. Navigate - The context graph engine determines the current state, evaluates transition conditions, and selects the appropriate action.
2. Engage - The response generation model produces a reply, drawing on the agent's persona, current state guidance, selected memory, patient data from the world model, and the emotional context described below.
3. Execute - If the model calls tools, the engine executes them, feeds results back as tool result signals, and re-engages. This loop continues until a final text response is produced.

Act. The engine emits effects. The modality adapter executes each one according to channel capabilities. For voice, the voice timeline applies cut/navigate/engage within each turn to coordinate fillers, empathy pauses, and tool progress narration - the same three operations at a smaller scale.

The engine supports two processing styles. Streaming mode lets the voice adapter begin response generation and speech delivery without waiting for a complete text response. Completed-effect mode lets text, simulation, and API consumers receive materialized effects. Both use the shared navigation and tool-execution contract, while voice-specific timing and acoustic adaptation remain in the voice path.

Filler handling is channel-aware. Voice can play a short acknowledgement while work continues. Asynchronous messaging channels suppress voice-style filler and deliver results through their channel-specific completion flow. See Email for a non-live example.

Navigation can also be re-evaluated after a successful tool result rather than waiting for another user message. This occurs only for tool bindings configured for completion-gated navigation; the navigator may remain in the current state or select another valid state. See Action State Extensions.

Emotional Adaptation

When a voice session has usable emotion data, the engine can add that context through two paths. Simulations may also supply emotion signals for testing, but ordinary text messages do not produce acoustic evidence.

Per-message annotations. When acoustic evidence is available, the user message can retain the current detected emotion and valence alongside its transcript. These annotations are model-derived signals, not verified statements about the caller's internal state.

Session-level steering. Once the voice runtime has enough evidence, it can add a rolling summary to response prompts:

• Dominant emotion and trend - Is the caller improving, stable, or deteriorating?
• Adaptation instructions - Targeted guidance based on the caller's emotional quadrant (high-arousal negative callers need de-escalation; low-arousal negative callers need patience)
• Behavioral signals - Patterns like repeated interruptions, short response streaks, or extended silences that indicate disengagement or frustration independent of vocal emotion
• Call-phase urgency - After extended calls with deteriorating mood, the engine instructs the model to become more direct and resolution-focused
• Coherence warnings - When what the caller says and how they sound disagree, the engine flags the ambiguity so the model does not over-commit to a single interpretation

The combination gives the response model recent evidence and adaptation guidance. It does not make emotion classification definitive, and safety or clinical decisions should not rely on emotion inference alone.

Per-State Configuration (TurnPolicy)

Each context graph state can configure the pipeline independently. A medication verification state behaves differently than a general scheduling state - not because the reasoning logic changes, but because the state's turn policy tunes the pipeline for that context.

Five areas are configurable per state:

• Barge-in - Enable or disable caller interruptions. A greeting state can suppress caller barge-in for a configurable shield duration; transport failure or other session termination can still interrupt playback. A quick-answer state can keep barge-in enabled for faster turn-taking.
• Safety response - What happens when a safety rule fires. Options: stay in the conversation and respond with empathy, suspend the agent and route to an operator, or log an alert without interrupting.
• Context strategy - Choose full history or compact past state groups. A per-state threshold can activate compaction, and the engine can also ratchet to compact mode when prompt usage approaches the configured context limit.
• Tool controls - The state defines its actions. Turn policy can additionally block call forwarding entirely or after a configured number of turns.
• STT sensitivity (voice only) - End-of-turn thresholds and silence timeouts. Data collection states use higher thresholds and longer timeouts because callers pause between pieces of information. Quick-answer states use lower thresholds for faster responses.

Model Configuration

The platform supports separate model preferences for navigation and engagement. Navigation selects a path through the context graph; engagement handles tool calling and response generation. Configuring them independently lets teams tune cost, latency, and response quality, but model-backed navigation is not inherently deterministic.

Graceful Degradation

Selected failure paths have bounded fallbacks so an optional subsystem or a single model timeout does not automatically end a session.

Component	Failure	Fallback
Emotion detection	Connection failure or repeated receive errors	Continues without new emotion-derived steering
Navigation model	Timeout or unavailable configured model	Tries a configured fallback; otherwise remains in the current engageable state
Context pressure	Estimated prompt usage approaches the configured limit	Compacts past state groups and caps retained history

These paths reduce failure propagation, but they are not a zero-failure guarantee. Transport loss, exhausted fallbacks, session limits, or failures in required dependencies can still interrupt or end an interaction.

Voice Control Plane

Voice calls combine service, agent, workspace, and environment settings that control vocal identity and delivery. They do not use one universal override hierarchy. Each field family has its own resolution rule.

Setting Family	Resolution
Service pipeline tuning	A service value applies when set; otherwise the field uses its documented runtime default or supported fallback
Session provider	Service, then agent, then environment
TTS provider	Service, then agent, then workspace, then environment. Per-language maps check the exact language, base language, and multilingual key, with service, agent, and workspace precedence within each key
TTS provider configuration and model	Workspace provider configuration forms the base, then agent and service configuration override it. A service model is the final model override; unresolved values use the environment configuration. A selected per-language entry can replace the merged base provider configuration
Workspace voice settings	Provide voice identity, tone, speed, volume, language, domain vocabulary, sensitive topics, and other workspace baselines for the fields that consult them
Per-state TurnPolicy	Controls separate state behavior such as STT sensitivity, barge-in, safety response, and context strategy; it is not another provider-inheritance tier

When the emotion system has usable evidence, it can derive a turn-level delivery tone and prompt guidance. Navigation can select a turn-specific tone, an explicit workspace tone can take precedence over computed voice context, and provider defaults apply when no turn tone is available. This tone choice is separate from provider and TTS configuration resolution. See Emotion Detection and the Audio Pipeline.

Concurrency

Voice timeline signals are processed through a per-session mailbox, and live text conversations use a per-conversation signal queue with an ownership lease. These boundaries serialize the work each actor accepts while still allowing external tools, providers, and background tasks to run independently.

Queueing reduces re-entrant state changes and gives each consumer an ordered input stream. It does not make the complete system deterministic or replayable: concurrent producers can race to enqueue, networks can retry or reorder delivery, and model output can vary. Durable traces support inspection, but they are not a bit-exact replay artifact.

The same model applies within voice turns. Fillers, responses, empathy pauses, and tool progress narration are not separate subsystems competing for the audio stream. They are effects emitted by the same actor into a single timeline, scheduled by cut/navigate/engage at the turn scale. One actor, one timeline.

Modality Adapters

Each adapter handles the channel-specific concerns that the reasoning engine does not touch:

Adapter	Signal Production	Effect Execution
Voice	STT produces utterance signals; prosody analysis produces emotion signals; silence and barge-in detectors produce their respective signals	Respond effects stream through TTS with emotion-adaptive delivery; fillers play audio; pauses hold silence; terminate effects hang up after final speech
Text (SMS)	Incoming messages produce utterance signals	Respond effects send SMS messages; terminate effects end the session
Simulation	Test parameters inject utterance and emotion signals	Effects are captured as simulation evidence; tool writes use simulation source tagging rather than a separate database branch

New modalities can reuse the signal-and-effect contract through an adapter. Channel-specific delivery, lifecycle, authorization, and failure handling still require integration and testing.

Related sections - See Context Graphs for how the engine navigates problem spaces, Functional Memory for bounded cross-session context, and Voice Agent for voice-specific pipeline details. Dynamic Behaviors documents a separate Classic API capability.