Side-Effect Architecture

Dynamic behaviors in the Amigo system aren't limited to influencing conversational responses; they can also trigger specific side-effects that perform external actions. A single behavior activation can simultaneously initiate knowledge retrieval, execute complex tool calling sequences, perform deep reflection, modify context graphs in real-time, or integrate diverse data streams from external systems—all while maintaining a unified operational context.

The Broader Power of Side-Effects

The true power of side-effects extends far beyond simple auxiliary actions. Side-effects represent a fundamental mechanism for dynamically transforming the agent's operational capabilities in response to conversation context. These transformations can include:

• Knowledge Access Reconfiguration: Dynamically altering what information sources (Knowledge) are available and how they're queried, often based on Memory cues.

• Reasoning Pattern Modification: Switching between different cognitive frameworks (Reasoning) optimized for specific domains, influenced by Memory and available Knowledge.

• System Integration Pathways: Opening new channels between the agent and enterprise systems in real-time, as a result of M-K-R driven decisions.

• Problem-Specific Specialized Analysis: Activating domain-specific LLMs or analytical tools (Knowledge/Reasoning) for deeper specialized reflection, typically cued by Memory or unfolding context.

• Safety Protocol Activation: Implementing domain-appropriate safety responses (Reasoning) based on detected risks (from Memory/Knowledge).

• Context Graph Evolution: Creating entirely new pathways and exit conditions (modifying Reasoning structures) as conversation context (Memory) and active Knowledge shift.

• Multi-Modal Integration: Enabling access to different sensory or data modalities (influencing Memory and Knowledge input) depending on context.

This expansive capability makes side-effects the crucial bridge between conversation (the M-K-R process in action) and tangible outcomes, enabling agents to adapt not just what they say, but their entire operational framework based on evolving conversational context and the dynamic interplay of Memory, Knowledge, and Reasoning.

Types of Side-Effects

Context Graph Modification

Every time a dynamic behavior is selected the system modifies the context graph, which can fundamentally alter how the agent operates in real-time.

These modifications may include exposing new tools that weren't previously available, initiating hand-offs to external systems when specialized processing is required, introducing new exit conditions that change how the conversation might conclude, or enabling reflection and self-modification capabilities. For example, when a conversation shifts toward financial risk discussion, a compliance-focused behavior might automatically add verification exit conditions to the context graph, ensuring proper protocols are followed before proceeding further. This adaptive framework allows the system to reconfigure itself based on conversation dynamics.

For example:

User: "I've been feeling really dizzy and nauseous since increasing my medication."

[INTERNAL: System detects potential serious side-effect and modifies context graph to add safety verification]

Agent: "Dizziness and nausea can be concerning side effects.

I'd like to check a few important details:
- When did these symptoms start?
- How severe is the dizziness?
- Have you been able to stay hydrated and maintain your normal activities?

This information will help determine appropriate next steps."

[INTERNAL: Context graph modified with new exit condition - if symptoms are severe, transition to medical support referral pathway]

Tool Integration

Triggered behaviors can dynamically invoke real-time data tools based on the evolving conversation context. When the system detects relevant conversational patterns, it can either force tool activation or simply make tools available through context graph modification. For instance, if a user mentions they are feeling tired after their workout yesterday, the appropriate behavior might automatically trigger heart rate trend analysis from their connected wearable device data without requiring the user to specifically request this information. This creates a more seamless experience where tools are invoked based on behavioral context rather than explicit commands.

For example:

User: "My running pace hasn't improved in two weeks even though I'm training consistently."

[INTERNAL: Behavior triggers performance trend analysis tool]

Agent: "Looking at your running data, I can see you're actually following a common training pattern.

Your overall trend shows steady progress with these natural plateaus along the way.

The data shows your average pace improved by 45 seconds per mile over 6 weeks, which is sustainable, healthy progress.

These 10-14 day plateaus are actually normal adaptation periods where your body is consolidating gains."

Reflective Processing

Behaviors can trigger the generation of structured inner thoughts that capture the agent's reasoning process. This feature serves two purposes: enhancing transparency for users and creating detailed decision rationales that can be audited for continuous improvement.

When complex decisions are being made, especially in high-stakes domains, the system can generate explicit reasoning chains that document how and why specific recommendations were formulated. For instance, when discussing potential treatment options, a clinical reasoning behavior might generate a structured reflection that documents the considered factors, potential contraindications, and evidence-based rationales behind the suggestions. These reflections provide visibility while also creating valuable data for system refinement.

Domain-Specific Specialized Reflection

A particularly powerful application of reflective processing is the ability to trigger specialized, domain-specific LLMs that can provide deeper expertise in highly technical areas. Rather than relying solely on the general agent's capabilities, side-effects can activate purpose-built models optimized for specific domains like medical diagnosis, legal analysis, or financial compliance. These specialized reflections can then be integrated into the agent's reasoning process, significantly enhancing the depth and accuracy of responses.

For example:

[INTERNAL REFLECTION: User has mentioned third consecutive day of severe headache with medication. Pattern indicates potential adverse reaction requiring medical attention rather than normal side-effect. Will acknowledge concern, gather minimal essential information, and direct to appropriate medical support per safety protocol.]

Agent: "A severe headache lasting three days while on this medication requires medical attention.

This isn't typically expected and should be evaluated by a healthcare provider.

Please contact your prescribing physician today or seek urgent care if the headache is worsening."

Enterprise Integration

Side-effects can be used to connect conversational interactions to broader organizational systems and workflows. These integrations enable the agent to initiate a variety of operational functions based on conversation content and context.

For example, dynamic behaviors can kick off asynchronous metric generation to provide insights to system administrators, send webhook notifications when conditions are detected that require attention, or enable seamless handoffs to human experts when conversations exceed the agent's capabilities. For example, when detecting potential medical concerns in a user's language patterns, a specialized behavior might trigger a clinical assessment process while simultaneously preparing a comprehensive handoff package for healthcare professionals. This integration layer transforms the agent from an isolated conversational entity into an integrated component of larger enterprise systems.

For example:

User: "I've been having chest pain and trouble breathing since starting this program."

[INTERNAL: System detects potential medical emergency, triggers healthcare provider alert]

Agent: "Chest pain and breathing difficulties require immediate medical attention.

Please call emergency services (911) right now or go to the nearest emergency room.

I've also notified the healthcare team, and they'll follow up with you directly.

Your safety is the top priority."

[INTERNAL: System has sent alert to healthcare team with conversation context and user contact information]

Cross-Graph Navigation

Side-effects can create entirely new pathways between different context graphs, enabling seamless transitions between distinct problem-solving spaces. This capability allows the system to maintain specialized, optimized graphs for different domains while creating dynamic bridges between them based on conversation flow.

For example, a conversation that begins in a general wellness context might detect potential cardiac issues, triggering a side-effect that creates a pathway to a specialized cardiovascular assessment graph with its own structured approach to symptom evaluation. This cross-graph navigation enables the agent to maintain domain-specific expertise while providing a unified user experience.

Complex Side-Effect Orchestration: Heart Issue Example

The full power of side-effects becomes apparent when multiple capabilities are orchestrated to handle complex, high-stakes situations. Consider this comprehensive example of how side-effects transform agent capabilities when a heart issue is detected:

User: "I've been having some pressure in my chest, especially when I climb stairs, and sometimes I feel short of breath."

[INTERNAL: Cardiovascular risk behavior activated, triggering multiple coordinated side-effects]

Side-Effect 1: Data Source Exposure
- System exposes specialized cardiology knowledge base to agent
- Provides access to cardiac symptom evaluation protocols
- Includes explanations of how to query these resources for specific patterns
- Agent now has access to specialized knowledge on exertional chest pain assessment

Side-Effect 2: Specialized Medical LLM Reflection
- Activates cardiology-specialized LLM for focused analysis
- Specialized model analyzes symptoms against cardiovascular risk patterns
- Generates structured clinical reasoning on potential cardiac vs. non-cardiac causes
- Provides probability-weighted differential diagnosis for agent consideration
- Reflection considers age, risk factors, symptom pattern, and exertional component

Side-Effect 3: External Alert Protocol
- Initiates tiered alert system based on risk assessment
- Prepares structured clinical summary for healthcare provider
- Generates secure notification to patient's care team with conversation context
- Creates trail of clinical reasoning for provider review

Side-Effect 4: Context Graph Modification
- Adds new exit conditions for urgent symptoms (e.g., acute severe pain, radiation to jaw/arm)
- Creates new pathway to emergency response protocol if urgent symptoms reported
- Modifies conversation structure to follow cardiac assessment guidelines
- Adds tool access for heart rate variability analysis if user has connected monitoring device
- Establishes checkpoints requiring user acknowledgment of recommendations

Agent: "I notice you're experiencing chest pressure with exertion and shortness of breath. These symptoms require careful attention.

I'd like to understand a few important details:
- Is the chest pressure/pain severe right now?
- Does it radiate to your jaw, arm, or back?
- Have you ever had heart problems before?
- Do you have any known risk factors like high blood pressure or diabetes?

These symptoms could potentially be related to your heart and should be evaluated by a healthcare professional. While we're talking, I've also notified your care team about these symptoms."

[INTERNAL: Agent now operating with specialized cardiological knowledge, following clinical assessment protocol, with new exit conditions for emergency scenarios, and healthcare team notification in progress]

This example demonstrates how side-effects transform the agent from a general conversational assistant into a specialized cardiac assessment tool with external integration capabilities, all triggered dynamically based on conversation content. The agent can now access specialized knowledge sources, leverage domain-specific reasoning models, connect to external healthcare systems, and follow structured clinical protocols—none of which were active before the cardiovascular risk was detected in conversation.

Unified Applications

Combining different side-effects significantly expands the capabilities of dynamic behaviors. By connecting dialogue to external tools, adapting interaction frameworks in real-time, documenting reasoning processes, and integrating with enterprise systems, the side-effect architecture enables a more comprehensive and effective interaction system that bridges the gap between conversation and practical functionality.

See below for an example of the unified system in action:

User Context: Tony (previous knee injury, recovering from sports rehabilitation, using fitness tracking device)

1. Tony: "I'm feeling tired today after my workout"

2. Dynamic Behavior Selection:
   - "Exercise Safety" behavior selected based on injury history in user model
   - Triggers side-effect: Activity data integration from Apple watch
   - Initiates internal reflection to assess fatigue patterns 

3. Side-Effect Execution:
   - Retrieves yesterday's workout intensity and heart rate recovery metrics
   - Compares against baseline patterns for this user
   - Generates risk assessment score for potential overtraining

4. Context Graph Modification:
   - Adds injury prevention instruction to next action state
   - Creates new exit condition requiring safety acknowledgment
   - Records reflection in interaction log for continuity

5. Knowledge + Data Response:
   - Integrates foundational exercise physiology knowledge 
   - Incorporates real-time heart rate variability data
   - Applies rehabilitation context from user model
   - Delivers personalized recovery recommendation

Implementation Best Practices

To effectively implement Dynamic Behavior with side effects:

1. Identify Key Conversation Patterns: Analyze common user questions, concerns, and scenarios

2. Encode Expert Knowledge: Work with domain experts to create specific guidance for each pattern

3. Define Appropriate Side Effects: Determine when tools, integrations, or system modifications are needed

4. Craft Flexible Trigger Patterns: Design triggers that range from associative tags to specific mentions based on needs

   • Can use broad conceptual triggers (e.g., "exercise", "motivation", "recovery")

   • Can use highly specific triggers (e.g., mentioning particular workout types or training equipment)

   • Remember triggers are for relative ranking, not exact matching

5. Design Variable Instruction Specificity: Develop instructions that range from open-ended to highly structured

   • Can provide general guidance for areas requiring flexibility

   • Can include precise step-by-step protocols for regulated or sensitive areas

6. Prioritize and Layer Behaviors: Establish clear priority relationships between different behaviors

7. Test with Real Scenarios: Validate behaviors against real user conversations

8. Monitor and Refine: Continuously analyze effectiveness and update behaviors based on outcomes

By implementing dynamic behaviors with side effects, organizations can create agent experiences that combine the reliability of structured protocols with the adaptability and nuance of human experts, while seamlessly integrating with enterprise systems to provide enhanced functionality beyond conversation.