Voice Configuration
Per-service voice pipeline tuning for latency, fillers, response limits, barge-in sensitivity, and emotion control.
Every voice service has a voice_config object that controls how the real-time pipeline behaves during calls. It covers latency tuning, filler behavior, response length limits, barge-in sensitivity, emotion modulation, voice model and TTS provider selection, and tool access. If you leave voice_config null, the service inherits balanced defaults from the workspace and environment.
The design principle is straightforward: each field is optional. A null field means "use the default from the layer above." You only set the fields you want to override.
Field Reference
Latency
These fields control how quickly the agent responds after the caller stops speaking.
tts_model
"sonic-turbo" or "sonic-3"
inherited
TTS model. sonic-turbo targets ~40ms time-to-first-audio and is best for snappy conversations. sonic-3 targets ~90ms but produces higher-quality speech with better prosody on longer utterances. Null inherits the environment default.
max_buffer_delay_ms
int (200-1000)
500
How long the pipeline buffers generated text before sending it to TTS. Lower values make the agent start speaking sooner, but may produce choppier speech because each TTS chunk has less context.
eager_eot_threshold
float (0.0-1.0)
inherited
End-of-turn confidence threshold. Higher values make the system more aggressive about deciding the caller has finished talking. A value of 0.8 means "start responding when 80% confident the caller is done." Lower values reduce latency but increase the chance of cutting the caller off mid-sentence.
eot_timeout_ms
int
inherited
Hard silence timeout in milliseconds. If the caller stops speaking for this long, the system forces an end-of-turn regardless of the confidence score. Useful as a safety net for the confidence-based detection.
post_eot_pause_ms
int
200
Intentional breath after the caller stops speaking, in milliseconds.
transition_deadline_ms
int
800
Maximum silence after caller end-of-turn before a filler is injected, in milliseconds.
empathy_hold_ms
int
500
Intentional silence held for empathy moments, in milliseconds.
Fillers
Fillers are the short sounds or phrases the agent produces while the LLM generates the full response. They fill what would otherwise be dead air.
filler_style
"backchannel", "phrase", or "silent"
"backchannel"
Controls the agent's autonomous fillers. Only silent changes runtime behavior: it mutes the agent's autonomous backchannels (the navigation acknowledgment and tool-wait acknowledgments). backchannel and phrase are currently advisory: the per-tool wait style is driven by the context graph's progress hints, not this field. Explicit context graph progress phrases still play even under silent.
filler_vocabulary
list[str]
null
Custom words for the short acknowledgment played when navigation is skipped (e.g., ["Mm", "Yeah", "Mhm"]). Does not change tool-execution wait phrases - those come from progress_vocabulary or the context graph's progress phrases.
backchannel_delay_ms
int (400-1000)
400
How many milliseconds the pipeline waits before playing a backchannel when navigation is skipped (the caller said something that doesn't require a state change). Lower values make the agent feel more responsive; higher values feel more deliberate.
progress_vocabulary
list[str]
null
Custom phrases for the tool-execution wait filler (distinct from filler_vocabulary). Ignored when the active tool's context graph progress hint is deterministic - scripted progress phrases play verbatim in that case.
filler_cooldown_ms
int
2000
Minimum gap in milliseconds between consecutive fillers.
progress_interval_ms
int
3000
How often to produce progress audio during tool execution, in milliseconds.
Response
Hard caps on how much the agent says per turn. These are mechanically enforced by the pipeline (it truncates the output), not just prompted.
max_response_sentences
int
null (unlimited)
Maximum sentences per response. Set to 1 or 2 for voice services where brevity matters. The pipeline counts sentences by punctuation and cuts off after the limit.
max_response_words
int
null (unlimited)
Maximum words per response. Same mechanical enforcement. Useful when you want finer control than sentence-level truncation.
Barge-in
Barge-in is when the caller starts speaking while the agent is still talking. These fields control how sensitive that detection is.
barge_in_min_speech_s
float
0.25
Minimum duration of caller speech (in seconds) before a barge-in triggers. The default of 0.25s lets short phrases ("yes", "no", roughly 200-400ms) interrupt the agent while the recognized-words gate keeps noise out. Duration alone never triggers a barge-in - the pipeline also requires recognized interim words from the STT engine, so breath sounds and background noise do not cause false interrupts. Raising the value (e.g., 1.0+) makes the agent feel like it ignores short interjections.
barge_in_cooldown_s
float
1.5
Cooldown period after a barge-in before another one can fire. Prevents rapid-fire interruptions where the caller and agent keep cutting each other off.
Emotion Modulation
min_tts_speed
float (0.5-1.0) or null
null (no floor)
Minimum TTS speed the empathy engine can set. The empathy system reduces speaking speed after detecting caller distress, but without a floor the reduction can make the agent sound unnaturally slow. Set this to prevent over-modulation while preserving the warmth response. Values are relative to normal speed (e.g., 0.85 means the agent never drops below 85% of normal speed).
Voice Model and TTS Provider
session_provider
"amigo", "gpt_realtime", or "gpt_live"
inherited
Overrides the voice model family at the service level. Null inherits the environment default (amigo).
tts_provider
"cartesia", "elevenlabs", or "groq"
inherited
Overrides the TTS provider at the service level. Null inherits from workspace/environment.
tts_config
object
null
Provider-specific configuration bag; keys depend on tts_provider.
language_providers
object
null
Per-language TTS provider map (up to 10 entries). Keys are BCP-47 language codes or multilingual.
Tools
forward_call_enabled
bool
false
Whether the forward_call tool is available during conversations. This is opt-in because call forwarding has cost and compliance implications. Even when enabled, the context graph's TurnPolicy can further gate forwarding (e.g., block_forward_call_after_turns).
forwarding
object or null
null
Per-service call-forwarding destination: { "forward_to": "<E.164 number>" }. Used by the forward_call tool as the static fallback destination when the agent does not pass an explicit number and no EHR location is configured.
Named Presets
Three presets cover common configurations. A preset sets multiple fields at once to a tested combination.
ultra_low_latency
sonic-turbo
200ms
backchannel
400ms
1 sentence
Quick intake, high-volume triage, yes/no flows
balanced
(default)
500ms
(default)
(default)
(unlimited)
General-purpose conversations
quality
sonic-3
500ms
phrase
(default)
(unlimited)
Sensitive topics, empathetic conversations where natural speech quality matters more than speed
Presets are a starting point. You can apply a preset and then override individual fields on top of it.
API Usage
Voice config is a field on the service model. You set it when creating or updating a service.
Setting voice config via PUT
The voice_config object is replaced wholesale, not merged. Merge behavior on service update applies only across other top-level service fields - within voice_config, whatever you send becomes the entire config. To change one field, send the full object: read the current config via GET, modify the field, and PUT the complete object back.
A partial body like this:
changes the filler style, but it also drops every omitted nullable field (tts_model, max_buffer_delay_ms, and so on revert to workspace and environment inheritance) and resets defaulted fields to their defaults (backchannel_delay_ms to 400, forward_call_enabled to false).
Reading current config
The service response includes the full voice_config object:
Fields that are null mean "inherit from the layer above" (see Configuration Cascade below).
CLI Usage
The forge CLI provides a shorthand for voice config operations.
Apply a preset
This sets all the fields in the preset at once.
Set individual fields
Read current config
Configuration Cascade
Voice behavior resolves through a four-layer cascade. Each layer can override the one above it. The first non-null value wins.
Here is what each layer controls:
Layer 1: Workspace Voice Settings set the baseline for all services in a workspace. Voice ID, tone, speed, volume, language, keyterms, and sensitive topics. Configured via GET/PUT /v1/{workspace_id}/settings/voice. See Workspaces.
Layer 2: Service Voice Config (voice_config on the service) overrides workspace defaults for pipeline behavior. This is where you set latency tuning, filler style, response limits, and barge-in sensitivity. A null field means "use whatever the workspace says."
Layer 3: Context Graph TurnPolicy can override behavior per-state. For example, a "crisis" state might set safety_response: stay_empathize and block_forward_call: true. TurnPolicy fields include barge_in_enabled, greeting_shield_s, safety_response, context_strategy, degradation_threshold, and block_forward_call. See Simulation Coverage for how these were tuned.
Layer 4: Channel Overrides on individual actions or states can override the objective, action guidelines, and filler suppression for specific channels (e.g., SMS versus voice).
In practice, most services only need Layer 1 (workspace) and Layer 2 (service voice config). Layers 3 and 4 are for advanced context graph tuning.
Turn-Level Emotion Selection
During each turn, the voice pipeline selects an emotional tone alongside the state transition, based on the caller's emotional state and the conversation context. The selected tone is applied to the entire response - the filler and the main response share it - so each turn is spoken with one consistent voice.
Available Emotions
The pipeline chooses from eight emotions:
friendly
Default warmth. General conversations, greetings, confirmations.
sympathetic
Caller is frustrated, upset, or sharing bad news.
calm
Sensitive topics, de-escalation, anxious callers.
enthusiastic
Positive outcomes, good news, caller is excited.
serious
Important information, disclaimers, medical details.
cheerful
Light conversations, small talk, closing on a positive note.
curious
Asking clarifying questions, exploring a topic.
content
Neutral satisfaction. Wrapping up, confirmations after a good interaction.
Because the tone applies to the whole utterance, the prosody stays consistent from the first filler word to the end of the response - there are no uncanny emotional shifts partway through a sentence.
The empathy system can override the turn-level selection at higher tiers. At Tier 2 (full empathy), the system forces warmth regardless of the selected emotion. At Tier 3 (hold space), filler is suppressed entirely and silence replaces speech.
Last updated
Was this helpful?