Continuous Improvement

Use production evidence, simulations, and versioned releases to improve agent behavior through an explicit, governed team workflow.

Every interaction can produce evidence about agent behavior: outcomes, metric scores, context graph paths, executed tools, latency, escalations, and operator involvement. Teams use that evidence to decide what to change, verify the change, and release it deliberately.

The platform supports this loop with analytics, evaluations, simulation, configuration versions, and deployment tools. It does not autonomously rewrite agent configuration, promote a candidate, or change world-model confidence policy from an observed correlation.

The Improvement Loop

1. Measure

Start with evidence that the platform already records:

• Conversation and run outcomes
• Configured evaluation results and metric trends
• Context graph states and executed tool calls when trace evidence is available
• Latency, escalation, and operator-participation signals
• Surface completion and outbound delivery outcomes

Production evaluations can run on demand or after completed conversations when eager evaluation is enabled. Dashboards and APIs expose results over configurable time ranges so teams can establish a baseline before making a change.

2. Investigate

Aggregate trends identify where to look; individual interactions explain what happened. Compare periods and relevant segments, then inspect transcripts, executed actions, and trace evidence for representative successes and failures.

Treat a correlation as a candidate explanation, not an automatic configuration instruction. A lower completion rate for one service might reflect a changed caller population, unavailable source data, a tool failure, or agent behavior. Confirm the cause before changing the agent.

3. Author a Versioned Change

Make the smallest change that tests the hypothesis. Depending on the finding, that may be a context graph transition, agent instruction, tool binding, service setting, evaluation definition, or test case.

Agents and context graphs use explicit versions, and version sets can pin the versions a service releases. Record related changes to non-versioned settings or evaluation definitions with the same release decision. The Developer Console, Platform API, SDK, and Agent Forge provide interfaces for these resources; none of them releases a change merely because a metric moved.

4. Validate Before Release

Use simulations to exercise the affected paths and compare the candidate against the baseline criteria. Add cases for newly discovered failure modes so the regression remains visible. Use real-audio tests when the change affects speech, timing, or media behavior, and use an isolated integration environment for connector or approval workflows.

Validation should cover both the target improvement and important guardrails. A faster scheduling flow is not an improvement if it lowers information accuracy or increases unsafe escalation behavior.

5. Release and Observe

An authorized user decides which tested versions and settings to release. After release, compare the same measures used for the baseline and watch for changes outside the target metric. Retained agent and context graph versions give the team a known configuration to restore deliberately if the release performs worse.

Multi-Objective Review

Healthcare workflows rarely have one success measure. Define an acceptance set that reflects the actual operating constraints, for example:

• Task completion meets its target
• Safety-critical evaluations remain within policy
• Information accuracy does not regress
• Caller experience remains acceptable
• Duration and cost stay within operating limits

The platform can compute and present these measures, but the team defines the trade-offs and release criteria. There is no autonomous optimizer choosing a configuration on the workspace's behalf.

What Does Not Feed Back Automatically

Analytics can show that one source, path, or tool correlates with stronger outcomes. That observation does not automatically:

• Change event confidence or source precedence in the world model
• Rewrite context graph transitions or tool order
• Adjust escalation thresholds
• Promote a version set
• Roll back a production release

Those controls remain explicit because an operational correlation is not enough evidence to change a clinical or safety policy.

Governance

The improvement workflow uses the platform's normal governance boundaries:

• Permissioned - Authorized roles control configuration and release actions.
• Auditable - Configuration changes and deployment actions retain actor attribution.
• Versioned - Candidate and released configurations remain distinguishable.
• Bounded - Safety requirements are release constraints, not variables to trade away for a higher average score.

Building Reusable Evidence

Improvement compounds when teams preserve what they learn:

• Convert production failure modes into permanent simulation cases.
• Keep baseline periods and release criteria with the change record.
• Reuse proven patterns across services only after validating them in the new context.
• Review metrics after each release using the same definitions and segments.

This creates an evidence base that improves future decisions without claiming that the system learns or deploys configuration changes on its own.

See Testing Overview for the validation workflow, Metrics and Quality for evaluation concepts, and Agent Forge for versioned configuration commands.