Call Intelligence

How Amigo scores every interaction - real-time profiles, post-call quality dimensions, the audio-native Voice Judge, decision traces, and playback timelines.

Every interaction - voice and text - produces a structured analytical breakdown covering emotion, risk, latency, safety, and outcome quality, all computed automatically. The result is a dataset that covers your entire operation, not just the calls someone happened to listen to. This page describes how each interaction is profiled, scored, and traced: real-time intelligence during the call, quality scoring after it, and the trace and playback tools used to understand exactly what happened.

Three Layers of Quality Analysis

Analysis happens in three passes. The first runs during the interaction. The second and third run after it ends.

Layer 1: Real-Time Intelligence

Seven structured profiles are computed while the interaction is still in progress.

Profile	What It Captures
Emotion	Dominant emotion, valence and arousal averages, peak negative moment, emotional shifts over time, final emotional trend
Risk	Composite risk score with contributing signals identified
Latency	Response time averages and percentiles, time-to-first-response, silence ratio
Conversation dynamics	Turn count, states visited, loop count, interruption count, completion reason
Tool performance	Success and failure counts per tool invoked during the interaction
Safety	Rule matches and escalation triggers fired
Operator involvement	Whether a human connected, time to connect, and resolution outcome

These profiles are available before the interaction ends. Monitoring dashboards and alerting rules can act on them in real time.

Layer 2: Post-Interaction Quality Scoring

After the interaction ends, a second pass scores quality across five dimensions on a 1-5 scale.

Dimension	What It Measures
Task completion	Did the agent accomplish what the caller needed?
Information accuracy	Were facts correct? Did the agent act on accurate data?
Conversation flow	Natural pacing, no awkward pauses or repetitions
Error recovery	Did the agent recover gracefully from confusion or unexpected input?
Caller experience	Overall experience based on tone, engagement, and interaction patterns

Each interaction also receives an outcome classification: succeeded, partially succeeded, failed, or abandoned.

Layer 3: Voice Judge

The third pass operates directly on the stereo call recording rather than structured session data, scoring ten audio-native dimensions - pronunciation, pacing, warmth, interruption handling, and more - that structured data cannot capture. See Voice Judge below for the full rubric and how results are served.

Composite Quality Score

Each interaction receives a composite quality score on a 0-100 scale, computed from the operational telemetry captured during the interaction. The score starts at 100 and deducts for specific quality signals: high latency, excessive silence, interruptions, agent loops, escalations, and tool failures. The Layer 2 dimension scores and the Voice Judge dimensions are reported alongside it as separate per-dimension metrics rather than folded into the composite.

Interactions are tiered based on this score:

Tier	Meaning
Excellent	No significant quality issues detected
Good	Minor issues that did not affect the outcome
Fair	Noticeable issues that may have affected the caller experience
Poor	Significant issues requiring review

The composite score is the primary metric for tracking quality over time and comparing performance across agents, configurations, and time periods. It is designed for dashboard filtering and trend analysis - you can filter calls by tier to focus review time on the interactions that need it.

Individual call scores are useful for reviewing specific interactions. Trends across thousands of calls are where you get operational visibility - see Call Quality Trends for quality distributions, escalation rates, and period-over-period comparison.

Key Moment Extraction

The system automatically identifies notable events - moments of elevated risk, emotional shifts, escalation triggers, tool failures - and tags them with timestamps.

Reviewers jump directly to what matters instead of listening to entire recordings. When a call scores poorly, the key moments tell you exactly where things went wrong.

Transcription Accuracy Feedback

Quality analysis feeds corrections back into transcription. When the scoring pass identifies likely transcription errors - a medical term misheard, a name consistently misspelled - it updates the speech recognition configuration.

Transcription accuracy improves over time for your specific vocabulary: medical terminology, provider names, local street names, insurance plan names. No manual tuning required.

Production and Simulation Intelligence

Call intelligence events are emitted for both production calls and simulation sessions. Simulation events carry a source tag that distinguishes them from production data, and include the simulation run identifier so metrics can be grouped by run. The metric evaluation pipeline deduplicates events per session and run before scoring, so retried or replayed completions do not inflate metric counts. Simulation-originated intelligence is persisted to the platform's analytical data store alongside production data, so metric evaluation, dashboards, and quality scoring work identically regardless of whether the conversation was a live call or a simulated test run. This unified persistence means simulation results feed directly into the same metric pipelines used for production quality monitoring.

Each voice call also produces a structured intelligence summary computed from session state at call end. These summaries capture operational telemetry that async quality scoring (which runs on recordings) cannot see: real-time emotion trajectories, engine response latency, tool invocation counts, and safety rule matches. The composite quality score (0-100) from this summary uses the penalty-based model described above and is the primary metric for dashboard filtering.

Turn-Level Analytics

Every conversation turn - whether voice or text - carries structured metadata that supports filtering, aggregation, and trend analysis without post-processing raw transcripts or nested data. Each turn record includes:

• Signal classification - what triggered the turn (end-of-turn speech, external event, tool completion, silence threshold)
• State tracking - the context graph state before and after the turn
• Effect classification - whether the agent responded, called a tool, or terminated the conversation
• Tool summary - name, success status, and duration of the primary tool call (when applicable)
• Modality - whether the turn occurred over a voice or text channel
• Empathy tier - the empathy classification for the turn
• Decision bearing - whether the turn resulted in a state change

These fields use a consistent vocabulary across voice and text channels, so analytics queries work uniformly regardless of the channel that produced the data.

Decision Trace

Every conversation turn is classified with structured signal and effect types, creating an explainability layer over agent reasoning. Each turn carries a signal kind (what triggered the agent's response - caller speech, tool result, state transition, barge-in) and an effect kind (what the agent did - spoke, invoked a tool, navigated to a new state, escalated). This classification is computed at read time from existing turn data, requiring no additional storage.

Decision events are emitted at the end of each call for analytics queries, producing three derived metrics: turns per call, tool invocation rate, and state transition rate. These metrics feed into the Metric Store and can be tracked over time to understand how agent behavior changes across configuration updates.

Agent Traces

Two debugging views provide full visibility into agent reasoning for any completed call:

• Execution trace - Per-turn log with the action taken, tools invoked, state transitions, detected emotions, inner reasoning thoughts, and response latency. Includes barge-in details showing which agent speech was interrupted and what utterances were discarded.
• Prompt trace - Full LLM prompts (system prompt, conversation history, tool definitions) for each turn. These are emitted per-turn as events, making them queryable for analysis across calls.

Equivalent traces exist for simulation sessions, so the same debugging tools work during testing.

Call Trace Analysis

Audio-native trace analysis provides deep intelligence on completed calls. Unlike the real-time profiles (which analyze structured data during the call), trace analysis operates on the full audio recording after the call ends and returns:

• Emotional arc - How the caller's emotional state evolved throughout the conversation
• Key decision moments - Critical points with causal attribution explaining why the agent made each decision
• Component attribution - Root cause analysis identifying which subsystem caused each issue (speech-to-text, navigation model, emotion detector, state machine, tool executor, prompt logic, or turn taking), with structural evidence from the execution trace
• Counterfactual analysis - What would have happened if the agent had taken a different action
• Coaching recommendations - Specific suggestions for improving agent configuration
• Interaction dynamics - Patterns in conversational flow, turn-taking, and engagement

Component attribution turns "the agent gave a wrong answer" into "the navigation model failed to process the caller's response despite high transcription confidence." This makes it actionable - you know which component to tune instead of guessing.

Analysis is computed asynchronously after call completion. The API returns a pending status while processing is in progress.

Call Playback Timeline

The call playback timeline provides a canonical, lane-based visualization of everything that happened during a call. Every event - speech, tool calls, escalations, fillers, silence - is organized into parallel lanes with actor attribution, creating a synchronized multi-track view of the interaction.

Lanes and Actors

Each timeline is divided into lanes - parallel tracks that group events by actor. The standard lanes are:

Lane	Actor Kind	What It Contains
Agent	agent	Agent speech segments, fillers, greetings
Caller	human	Caller speech, barge-in events
Operator	operator	Operator speech after escalation join
System	system	State transitions, emotion shifts, safety events
Tool	tool	Tool invocations with start/end timing

Every segment carries an actor with a kind (agent, human, operator, system, tool), a role (agent, caller, operator, runtime, state, tool), and a display label. This attribution lets UIs render each event in the correct lane and identify who or what caused it.

Segment Types

Segments are typed to distinguish different kinds of activity within each lane:

• Speech: agent_speech, caller_speech, operator_speech - audio segments with transcript text
• Interaction: barge_in, filler_hesitation, filler_phrase, backchannel - conversational micro-events
• Execution: tool_call, state_transition, escalation - system actions with metadata
• Silence: silence, hold - gaps in activity, distinguished by intent

Shared Timebase

All segments reference a shared timebase that anchors the timeline to a canonical reference point - either the media start, the call start, or a synthetic origin for multi-call timelines. Start and end times are expressed in seconds relative to this timebase, so all lanes are synchronized regardless of when each participant joined the call.

For calls that span multiple legs (transfers, conference joins), the timebase provides a unified coordinate system across the full interaction, and segments carry parent call references so the UI can show the complete timeline as a single continuous view.

The call timeline is accessible through the Platform API alongside call detail and call intelligence endpoints. See the Audio Pipeline for how voice timing parameters (filler cadence, end-of-turn detection, barge-in thresholds) shape the raw timeline events.

Voice Judge

The Voice Judge is an audio-native quality evaluator that scores every call across ten dimensions. Unlike the post-interaction quality scoring (which analyzes structured session data), the Voice Judge operates directly on the stereo call recording and produces per-dimension scores with structured evidence.

Ten Evaluation Dimensions

Dimension	What It Measures
Latency and Dead Air	Response timing and silence management
Pronunciation	Speech clarity and correctness, including medical terms
Clarity	Intelligibility of speech
Filler and Silence	Graceful tool-call acknowledgments without dead air
Interruption Handling	Clean barge-in detection and recovery
Audio Consistency	Volume and pitch stability without cutoffs
Pacing	Conversational tempo and pauses
Warmth and Tone	Tonal appropriateness to the caller's state
Accent Quality	Language match and accent naturalness
Voice Identity	Consistent agent voice and persona

Each dimension produces a score (0.0 to 1.0), a severity level (none, flag, warning, or critical), and an evidence string citing specific moments from the audio.

How It Works

The Voice Judge runs on a scheduled cadence, processing calls that have completed since the last run. Short calls (under 5 seconds) are skipped to avoid scoring dropped connections and test pings. Each call's stereo audio is evaluated against a structured rubric, and results are persisted for retrieval.

Results are available through two paths:

• Per-call detail - a dedicated endpoint returns recent scores for a specific service, with the full evidence JSON for UI drill-down into individual calls
• Aggregate metrics - the ten dimensions are registered as built-in metrics in the Metric Store, so workspace-level trends flow through the same dashboard and alerting infrastructure as all other metrics

The per-call endpoint is designed for investigation ("what went wrong on this specific call?"). The aggregate metrics are designed for monitoring ("is pronunciation quality trending down this week?").

Developer Guide - For API access to trace analysis and audio-native scoring, see Call Trace Analysis and Voice Judge in the developer guide.