Agent Forge CLI

Agent Forge CLI for syncing, versioning, and promoting agent configuration, with current Go and legacy Python build coverage.

Agent Forge is the CLI tool for managing agent configurations on the Amigo platform. It lets you create, update, version, and promote agent components programmatically rather than through the web interface.

Agent Forge treats agent configurations as code. You sync configurations to local JSON files, make changes, and push them back to the platform. This gives you version control, reproducibility, and the ability to script deployment workflows.

Choosing a Build

Agent Forge ships as two builds that share the same forge command and most of the same command surface:

• Go CLI (current, recommended) - A single self-contained binary with no runtime dependencies. This is the source of truth for the forge CLI going forward. New workspaces and new automation should use this build. The Go CLI officially supports only the Amigo Platform API - it drives all remote operations through the forge platform ... command tree. It does not officially support the Classic API (the legacy backend); use the Python CLI for Classic API workflows.
• Python CLI (legacy) - The original Poetry-managed Python tool. It supports both the Platform API and the Classic API, and it carries a superset of commands - a handful of analytics, quality, and reporting commands have not yet been ported to the Go binary - but it is being retired for external use and now mainly serves internal Amigo workflows. Use it if you need the Classic API or one of the legacy-only commands listed in its tab below.

Both builds understand the same entity model, the same --env flag, and the same .env.platform.<env> configuration for the Platform API. The tabs below document each build; unless a section says otherwise, the Platform API command syntax is identical across builds.

To tell which build you have, run forge version (Go prints a stamped binary version; the Python tool runs from a git checkout).

What Agent Forge Manages

Across its two builds, Agent Forge manages the following configuration and testing resources. Coverage differs by build: the current Go CLI targets Platform API resources, while the legacy Python CLI retains Classic API workflows and legacy-only analytics commands. The tabs below identify the supported command surface for each build.

• Agents: Persona, background, directives, and communication style
• Context graphs: Problem structure, states, transitions, and safety boundaries
• Metrics: Evaluation criteria, scoring rubrics, and custom metric definitions
• Personas: Synthetic user profiles for simulation testing (the primary way to manage personas)
• Scenarios: Test situations for simulation testing
• Services: Link an agent and context graph into a deployable unit
• Harness context: Inspect the supported service bootstrap context for customer integrations
• Tools: Versioned code packages (called Actions in the conceptual docs)
• Unit test sets: Groups of tests with success criteria
• Unit tests: Individual test cases
• User dimensions: Attributes that segment users for evaluation and analysis

Go CLI (current)

The Go build is the recommended, go-forward CLI. It ships as a single binary and covers the full agent-building and platform-management workflow.

The Go CLI officially supports only the Amigo Platform API. All remote operations go through the forge platform ... command tree. Classic API (legacy backend) workflows are not officially supported in this build - use the Python CLI tab for those.

Installation

Agent Forge ships as a single binary with no runtime dependencies. On macOS, the installer detects the architecture, downloads the correct binary, verifies the SHA256 checksum, and places it on your PATH.

curl -fsSL https://forge.platform.amigo.ai/install.sh | sh

Pre-built targets cover macOS (Intel and Apple Silicon) and Linux (amd64 and arm64). The public installer and the forge update self-updater currently target macOS. For Linux, obtain the binary through the release channel provided to your organization.

After installation, configure credentials for your workspace:

# Create environment file
cp .env.platform.example .env.platform.<your-env>
# Edit with your Platform API URL, workspace ID, and API key or identity URL

# Verify
forge auth status --platform --env <your-env>

You can also manage credentials as reusable profiles instead of environment files with the forge platform config command group (add, use, list, show, doctor, remove, import-env, path).

Authentication

The Go CLI authenticates against the Platform API. Pass the --platform flag on the auth commands to target the Platform identity service. Two methods are supported, selected automatically based on the environment configuration: device code login through the identity service, and static API keys.

Device Code Login (Recommended)

Device code authentication follows RFC 8628. When you run forge auth login --platform, the CLI requests a device code from the identity service, displays a short user code, opens your browser to an approval page, and polls for authorization. You verify that the code shown in the browser matches the code in your terminal and approve the request. These flows work in headless environments, SSH sessions, and CI pipelines where a browser cannot be opened inline.

The approval page enforces that your browser session is scoped to the same workspace the device code targets - if your session is scoped to a different workspace, the page redirects you to workspace selection first, and after choosing the correct workspace you are returned to the approval page automatically. If you are not signed in at all, the sign-in flow preserves the approval page as the return destination through authentication and workspace selection, so you land back on the approval page without needing to re-open the CLI link. A session scoped to the wrong workspace or a session without any workspace selected cannot approve the code.

The platform identity device code flow is workspace-scoped end to end. When you initiate a login, Forge sends the configured workspace ID along with the device code request, and the identity service binds the code to that workspace - the approver in the browser must hold a session scoped to the same workspace, and the resulting CLI token is scoped to it. This workspace enforcement applies at both the approval step and the token exchange step, ensuring that credentials are always tied to the intended workspace and preventing cross-workspace token misuse.

Once approved, Forge receives an access token and refresh token automatically - no manual token management required. Platform identity tokens are stored in a workspace-keyed file cache under the operating system's user configuration directory; Forge creates the cache directory with owner-only access and writes token files with mode 0600. An expired access token is refreshed with the stored refresh token without requiring re-authentication when refresh succeeds. Device code login replaces the need for a static API key for interactive CLI use.

API Key

Static bearer token authentication. Generate an API key from Amigo Console under Developer > API Keys and add it to your environment file. Platform API keys expire after a configured 1-90 days; Forge defaults new keys to 30 days. Rotate CI/CD credentials before expiry and delete keys that are no longer needed.

Environment Configuration

Platform API authentication reads from .env.platform.<env> (preferred) or falls back to .env.<env>. The following variables control the platform auth path:

Variable	Required	Description
PLATFORM_API_URL	Yes	Platform API URL for the target environment
PLATFORM_WORKSPACE_ID	Yes	Workspace to authenticate against
PLATFORM_API_KEY	One of these	Static API key (no login required)
IDENTITY_URL	One of these	Platform identity service URL (enables device code login)

If PLATFORM_API_KEY is set, Forge uses it as a static bearer token. If IDENTITY_URL is set instead, Forge uses the device code flow via forge auth login --platform.

Forge-native configuration fields are automatically translated to platform-native equivalents at deployment time. For example, audio filler phrases defined in Forge tool specs are converted to the platform's progress hint format, so agents configured through Forge work without manual migration.

Auth Commands

# Platform API login (platform identity device code)
forge auth login --platform -e myorg

# Check auth status
forge auth status --platform -e myorg

# Clear cached credentials
forge auth logout --platform -e myorg

Always pass --platform on the Go CLI. The --platform flag is available on login, logout, and status. Without it, the auth commands operate on Classic API (legacy backend) credentials, which the Go build does not officially support.

Command Groups

The sections below cover the Agent Forge command surface, one command group at a time. Most commands support --json for structured output, and --env selects the target environment, enabling integration with scripts and CI/CD pipelines. (forge validate, which runs against local files, accepts --env but not --json.)

Sync and Deployment

Pull from the platform - Read entities down from the Platform API with the per-resource get and list commands, then edit them as JSON on disk:

# List and fetch entities from the Platform API
forge platform agent list --env myorg
forge platform agent get <agent-uuid> --env myorg
forge platform context-graph get <context-graph-uuid> --env myorg
forge platform service get <service-uuid> --env myorg

Push to the platform - Push local changes back with forge platform push (see Bulk Push below). Before applying changes, Agent Forge shows exactly what will be modified so you can review before confirming.

The Go binary still carries the older sync-to-local / sync-to-remote commands, but they operate against the Classic API (legacy backend), which the Go build does not officially support. For Platform API work, pull with forge platform <entity> get / list and push with forge platform push. For Classic API sync, use the Python CLI.

Pre-Push Validation - Agent Forge validates context graphs before pushing to the platform and surfaces warnings for common authoring mistakes. Validation runs automatically during forge platform push, and you can run it on local files at any time with forge validate (no auth required).

The canonical value lint detects phone numbers, email addresses, and URLs hardcoded into context graph state prose. Inline canonical values cause silent data drift - when graphs are cloned or updated, hardcoded digits can be accidentally mutated, and the agent reads incorrect information to callers.

The validator scans prose fields in every state (descriptions, instructions, boundary constraints, exit conditions, and action descriptions) and emits a warning for each match, identifying the state, field, and value. It catches phone numbers in digit form (e.g., 555-010-1234), phone numbers in spelled-out TTS form (e.g., "five five five zero one zero..."), email addresses, and URLs.

To fix a warning, move the canonical value into structured context - such as a location entity in the world model or a workspace setting - and reference it abstractly in the state prose.

Environment Support - Agent Forge supports separate staging and production environments. Changes are deployed to staging first, validated through testing, and then promoted to production. Platform push operates on agents, context graphs, and services, laid out per environment:

project/
  local/
    staging/
      entity_data/
        agent/
        context_graph/
        service/
      .platform_id_map.json
    production/
      entity_data/
        (same structure)
      .platform_id_map.json

Bulk Push - Push local entity configurations to the Platform API in a single operation:

forge platform push --all --env myorg --apply

Supports selective push by entity type (-e agent, -e context-graph, -e service). The Go build maintains a local-to-platform UUID map (local/<env>/.platform_id_map.json) so repeated pushes update the same platform entities.

Agent Building

The forge platform command group provides broad CLI coverage for common Platform API workflows, enabling agent building and workspace management without the web interface. Build a complete agent from the CLI in four steps:

1. Create agent and agent version with identity, background, and behaviors
2. Create context graph and version with states, transitions, and exit conditions
3. Create service linking the agent and context graph together
4. Add skills (optional) for LLM-backed micro-agent capabilities

forge platform agent create --name "My Agent" --env myorg
forge platform agent create-version <agent-uuid> --file agent-version.json --env myorg
forge platform context-graph create --name "My Context Graph" --env myorg
forge platform context-graph create-version <context-graph-uuid> --file context-graph-version.json --env myorg
forge platform service create --name "My Service" --agent-id <agent-uuid> --context-graph-id <context-graph-uuid> --env myorg

CLI support is organized into these resource groups:

Resource Group	Commands
Core	workspace, agent, context-graph, service, version-set, skill
Voice & Text	conversation, run, voice-settings, voice-check
Data	integration, external-integration, function, fhir, workspace-table, data-query
Surfaces	surface
Testing	simulation, sim, tool-test, regression, agent-run
Access & Identity	api-key, role-grant, role-assignment, external-role, use-case
Operations	config, audit, session, triggers

Harness Context and Retired Framework Commands

The Platform API retains a read-only service harness-context capability for inspecting the selected identity, reference instructions, world scope, tool descriptors, and guardrail summary. It is bootstrap context for an integration, not an endpoint that launches a customer framework agent or retrieves patient context.

forge platform agent-run harness-context --service-id <uuid> --env myorg

The standalone native-agent definition registry and hosted framework-run service have been retired. Some Forge releases still expose agent-definition commands and agent-run create or get, but their former Platform endpoints are no longer available. Do not use those commands for new deployments. Use the supported conversation lifecycle for hosted interactions and the unified run commands to inspect retained run evidence.

Voice Configuration Presets

forge platform service voice-config can read the current configuration, apply an inline JSON body, or apply a named preset. The CLI recognizes ultra_low_latency, balanced, quality, gpt_realtime, and gpt_live. The gpt_realtime preset selects OpenAI Realtime speech-to-speech; provider, model, and regional availability are still enforced by the platform.

forge platform service voice-config <service-id> --get --env myorg
forge platform service voice-config <service-id> --preset gpt_realtime --env myorg

Simulation

Use forge platform simulation for the supported branch-and-bound coverage surface. forge platform sim bridge remains a shortcut to the same bridge workflow, and forge platform sim config-to-policy maps a configuration object to turn-policy fields. Legacy configuration-space commands such as sim create, sample, evaluate, status, points, summary, and complete target retired endpoints and should not be used.

Simulation Coverage

The forge platform simulation command group manages branch-and-bound simulation coverage runs that systematically explore context graph state space.

Command	Description
forge platform simulation run create	Create a new coverage run for a service
forge platform simulation run list	List coverage runs for a service
forge platform simulation run complete	Complete a run
forge platform simulation session create	Create a session within a coverage run
forge platform simulation session step	Step a session forward with a simulated user message
forge platform simulation session fork	Fork a session into children at a decision point, each with a different message
forge platform simulation session score	Score a session against configured metrics
forge platform simulation graph show	Retrieve the coverage knowledge graph with topology overlay and ghost nodes
forge platform simulation graph paths	List observed paths through the coverage graph

See Simulation Coverage for conceptual background.

Simulation Caller and Entity Context

forge platform simulation session create accepts --caller-id for a simulated caller phone number in E.164 format and --entity-id for direct binding to a known world entity. forge platform sim bridge accepts --entity-id but not --caller-id. Direct entity context is useful for regression tests against a known patient or account fixture. Omit the context flags to simulate an unknown caller.

# Bridge scenarios against a known entity
forge platform sim bridge --service-id <uuid> -o "Test known entity flow" --entity-id <uuid> --env myorg

# Branch-and-bound coverage session with entity context
forge platform simulation session create --run-id <uuid> --service-id <uuid> --entity-id <uuid> --env myorg

Tracked Platform Simulation Sessions

The forge platform sim group also drives tracked platform simulation sessions used by the Agent Readiness workflow:

Command	Description
forge platform sim smoke-test	Single-turn sanity check via a tracked platform session
forge platform sim session-create	Create a tracked simulation session (accepts --caller-id / --entity-id)
forge platform sim run-create	Create a tracked simulation run
forge platform sim run-list	List simulation runs with filtering
forge platform sim run-complete	Mark a simulation run as complete

Text Conversation Testing

The forge platform conversation command group tests text conversations through the REST API without a phone or browser. These commands are useful during initial setup, after configuration changes, or as part of a deployment validation pipeline.

Command	Description
forge platform conversation list	List conversation runs through the unified Runs view; use run list for new automation
forge platform conversation get	Retrieve conversation metadata, transcript, and optional tool calls
forge platform conversation create	Create a durable text conversation for a service and optional entity context
forge platform conversation send-message	Send a message, wait for the final reply when possible, and optionally evaluate it inline
forge platform conversation poll	Claim a completed background reply after a pending send
forge platform conversation close	Close a text thread so the next inbound message starts a new conversation

Create a durable text conversation, then send user messages through the REST turns endpoint and display the agent's response:

# Create the conversation
CONV_ID=$(forge platform conversation create \
  --service-id <uuid> \
  --env myorg \
  --json | jq -r .id)

# Send the first message
forge platform conversation send-message \
  --conversation-id "$CONV_ID" \
  --message "What appointments are available tomorrow?" \
  --env myorg

# Continue the same conversation
forge platform conversation send-message \
  --conversation-id "$CONV_ID" \
  --message "How about 2pm?" \
  --env myorg

# With patient context
forge platform conversation create --service-id <uuid> --entity-id <uuid> --env myorg

send-message requires an existing conversation ID. Use conversation create first and pass the returned ID on subsequent calls to continue the same conversation thread.

Inline Reply Evaluation

send-message can evaluate a final reply in the same command. Repeat --must-contain or --must-not-contain for fast local checks. Repeat --judge-criteria for model-based checks against natural-language requirements; judge checks require configured judge credentials and --i-understand-pii because the reply may contain sensitive data.

forge platform conversation send-message \
  --conversation-id "$CONV_ID" \
  --message "I need to move tomorrow's appointment" \
  --must-contain "appointment" \
  --must-not-contain "I cannot help" \
  --judge-criteria "The reply offers a concrete next step" \
  --i-understand-pii \
  --env myorg

Forge requests a final, filler-free reply. If background work is still pending, it does not grade the interim acknowledgement; use conversation poll to claim the completed reply.

The Go binary still registers forge platform conversation text-ws-smoke as a deferred placeholder and does not run the WebSocket test. Use the REST send-message flow above, connect through the documented Sessions WebSocket directly, or use the legacy Python command described in its tab.

Unified Runs

forge platform run is the canonical read surface across conversation runs and framework runs. It uses the same channel-neutral run identifiers as the Developer Console Runs page.

Command	Description
forge platform run list	List runs with kind, channel, and status filters plus continuation-token pagination
forge platform run summary	Show aggregate totals, live counts, status counts, and kind counts
forge platform run get	Retrieve one run by its channel-neutral run ID

forge platform run list --kind conversation --channel voice --status live --env myorg
forge platform run summary --kind conversation --env myorg
forge platform run get <run-id> --env myorg

Use conversation for conversation-specific create, send, poll, and close operations. Use run to inspect conversations and retained framework history. Hosted framework-run dispatch is retired; agent-run harness-context remains a separate read-only capability. See Runs in the developer guide.

Tool Testing

The forge platform tool-test commands let you test context graph tools without making phone calls:

Command	Description
forge platform tool-test resolve	List available tools for a service with input schemas
forge platform tool-test execute	Execute a tool with custom parameters and optional dry run mode

Trigger Management

The forge platform triggers command group manages action automations that run on a schedule, in response to supported platform events, or through a manual fire.

Command	Description
forge platform triggers create	Create a trigger with an action binding and schedule or supported event type
forge platform triggers list	List triggers with active/inactive filtering
forge platform triggers get	Get trigger details including next fire time
forge platform triggers update	Update trigger configuration
forge platform triggers delete	Delete a trigger
forge platform triggers pause	Pause a trigger's schedule
forge platform triggers resume	Resume a paused trigger
forge platform triggers fire	Manually fire a trigger for testing
forge platform triggers runs	View trigger execution history

Scheduled triggers use cron expressions. Event-based triggers match a supported world-model event and can narrow matches with an event filter. Manual fires use the same durable execution path, which makes them useful for testing a trigger before activation. See Outbound for how triggers fit into automated contact patterns.

Platform Functions

The forge platform function command group manages platform functions - declarative SQL, Python, AI, and table-valued (UDTF) functions that agents can call mid-conversation. Table-valued functions return rows rather than a single value.

Command	Description
forge platform function register	Register a new platform function with its definition and metadata
forge platform function list	List all registered functions in the workspace
forge platform function test	Execute a function with test parameters and inspect the result
forge platform function delete	Remove a function registration
forge platform function query	Run an open-scope SQL query against workspace data
forge platform function catalog	Display the full function catalog with signatures and descriptions
forge platform function sync	Sync function definitions between local files and the platform

See Platform Functions for conceptual background.

CLI Updates

The Go binary ships a built-in self-update command (macOS) that upgrades the binary in place:

forge update                   # download and install the latest release
forge update --check           # report whether an update is available; install nothing
forge update --version 1.4.2   # pin an exact release (up- or down-grade)

Here forge update resolves the target version, downloads the release for your Mac's architecture from the same Forge CDN the installer uses, verifies its SHA256 checksum, sanity-checks the downloaded binary before swapping it, and atomically replaces the running binary in place. The download always comes from the immutable, version-pinned release path, so a publish in progress can never mix generations mid-download.

Flag	Purpose
--check	Report whether an update is available and exit without downloading or writing anything.
--version <x.y.z>	Install this exact release instead of the latest, up- or down-grade. A downgrade below the installed version prompts for confirmation first.
--force	Reinstall even when already up to date, overwrite a locally built dev binary, and skip the downgrade confirmation prompt.
--yes, -y	Skip the downgrade confirmation prompt for non-interactive or scripted use.

If the install directory is not writable (for example, forge lives in /usr/local/bin), update fails with guidance to re-run as sudo forge update rather than silently escalating privileges. On Linux, reinstall from your organization's release channel to upgrade.

Python CLI (legacy)

The Python CLI is the legacy build and is being retired. Prefer the Go CLI (see the other tab) for all new work. Use the Python build only if you depend on one of the legacy-only commands listed below that has not yet been ported to the Go binary.

The Python CLI shares the Platform API command surface with the Go build - auth, validate, forge platform push, and the forge platform resource groups for agents, context graphs, services, skills, functions, conversations, tool tests, simulation, surfaces, and integrations. For those, follow the Go CLI tab; the syntax is the same. In addition, because the Python build also supports the Classic API, it offers the legacy sync-to-local / sync-to-remote workflow against the legacy backend. This tab documents what is different in the Python build.

Installation

The Python CLI is not a standalone binary - it runs from a git checkout with a full Python runtime, managed by Poetry.

1. Install Python 3.13 (the repo pins 3.13.5, e.g. via pyenv install 3.13.5).
2. Install Poetry 2.x.
3. From the repo root, install dependencies and activate the environment:

poetry install
$(poetry env activate)

Commands run as poetry run forge ... (or just forge ... inside the activated environment). Because the tool runs from a checkout, it must be invoked from the repository root.

Authentication

Unlike the Go build, the Python CLI supports both the Classic API (legacy backend) and the Platform API. The --platform flag selects which surface a command targets - omit it for the Classic API, pass it for the Platform API. Both use device code login and static API keys. Configuration is read from .env.<env> (Classic API) and .env.platform.<env> (Platform API) files. For the Platform-API authentication details (device code flow, environment variables), see the Go CLI tab.

Command-name differences

The command groups are largely identical to the Go build, with a few naming differences to watch for:

• The trigger group is forge platform trigger (singular) in the Python CLI, versus forge platform triggers (plural) in the Go CLI.

Legacy-only commands

The following commands are implemented in the Python build only. They are recognized but not yet functional in the Go binary, so if you rely on any of them, use the Python CLI until they are ported.

Analytics (forge analyze)

The forge analyze command group provides SQL-based exploration of workspace data directly from the CLI, replacing the need for external analytics tools.

Command	Description
forge analyze query	Execute ad-hoc SQL SELECT queries (inline or from file). Results are capped and queries are time-bounded.
forge analyze describe	Preview a query's output schema without executing it - useful for validating JOINs and checking column types.
forge analyze tables	List available tables in the workspace schema. Supports SQL LIKE patterns for filtering.
forge analyze schema	Describe a table's columns: names, data types, and comments.
forge analyze sample	Preview sample rows from a table (default 5, max 20).
forge analyze detail	Rich table metadata: row count, size, partitioning, column nullability, data freshness.
forge analyze profile	Profile a column's data distribution: cardinality, null rate, min/max values.
forge analyze catalog	Display the full data catalog reference offline without a database connection.

Pre-built analytics query templates cover common patterns like conversation volume, tool performance, and metric trends:

# List templates
forge analyze template list

# Run a template with parameters
forge analyze template run conversation-volume -P days=7

Insights (forge platform insights)

The forge platform insights command group provides conversational data exploration from the CLI, wrapping the platform's Insights Agent capabilities - workspace queries, schema metadata, and health digests.

Command	Description
forge platform insights sql	Execute a SQL query against workspace data and return formatted results
forge platform insights schema	Describe available tables, columns, and functions in the workspace schema
forge platform insights digest	Generate an AI-powered digest summarizing recent workspace activity, entity counts, and data quality signals
forge platform insights suggestions	Get suggested starter questions based on the workspace's data and recent activity

# Execute a SQL query against the workspace data warehouse
forge platform insights sql "SELECT ..." --env myorg

# Read SQL from a file
forge platform insights sql --sql-file my_query.sql --env myorg --json

# List available tables, columns, and functions
forge platform insights schema --env myorg

# Get a workspace health digest with entity counts and data quality signals
forge platform insights digest --env myorg

# Get suggested starter questions for exploring workspace data
forge platform insights suggestions --env myorg

Call Trace Analysis (forge platform trace)

The forge platform trace command group provides call trace analysis from the CLI, wrapping the platform's trace analysis capabilities - deep call understanding from the intelligence pipeline.

Command	Description
forge platform trace list	List call traces with filters for date range, service, quality score, outcome, and direction
forge platform trace get	Get detailed trace analysis for a specific call, including emotional arc, decision moments, component attribution, and coaching recommendations

# List recent trace analyses
forge platform trace list --env myorg

# Filter by outcome and lookback window
forge platform trace list --outcome failed --days 7 --env myorg

# Get detailed trace analysis for a specific call
forge platform trace get <call-sid> --env myorg

Trace analysis provides:

• Emotional arc - How caller sentiment evolved across the conversation
• Key decision moments - Critical points with quality assessment and causal attribution
• Coaching recommendations - Actionable improvements tied to specific call moments
• Counterfactuals - Alternative actions that could have changed the outcome
• Signal-response alignment - Whether the agent responded appropriately to caller signals
• Interaction dynamics - Turn-taking quality, rapport trajectory, and repair effectiveness

Trace output uses rich formatting with colored outcome indicators and structured digest sections for quick scanning of call quality issues.

Conversation Quality Check

The forge quality check command scans workspace conversations for agent behavioral issues - stuck loops, degenerate output, repetition, and other quality problems. It queries production conversation data directly and runs pattern-based detectors to surface problematic interactions.

# Scan last 24 hours
forge quality check <workspace-name>

# Wider window with message snippets
forge quality check <workspace-name> --days 7 --verbose

# Structured output for scripting
forge quality check <workspace-name> --json

Detector	What It Finds
Character degeneration	Repeated characters, low entropy output, stuttering patterns
Stuck agent loops	Agent repeats the same response while the caller changes topics
Repetitive patterns	High similarity across sliding message windows
Word salad	Incoherent output patterns like or-chains and excessive word repetition
Phantom success	Agent claims a tool call succeeded when the tool actually returned an error
Wrong tool inputs	A tool is called with parameters that do not match what the caller actually asked for
Ungrounded claims	The agent asserts capabilities or facts not supported by the configured entity definitions
Safety / PII	The agent leaks sensitive information or provides unsafe guidance

Some detectors (phantom mismatch, wrong tool inputs, ungrounded claims, and safety/PII) require an LLM key to run.

Results include conversation IDs, timestamps, detector names, and severity. Use --verbose to see the actual message excerpts that triggered each finding.

See Voice Simulation and Drift Detection for related quality monitoring capabilities.

Metrics Management (forge platform metrics)

The legacy Python CLI exposes workspace metric settings and current metric reads. Use only commands backed by the current Platform API:

Command	Description
forge platform metrics settings	View built-in and custom workspace metric definitions
forge platform metrics define	Replace the custom definitions supplied in a JSON document
forge platform metrics list	List current metric values
forge platform metrics catalog	List active built-in and custom catalog entries
forge platform metrics get <metric-key>	Read values for one metric
forge platform metrics trend <metric-key>	Read a metric time series

# View current metric settings
forge platform metrics settings --env myorg

# Define a metric
forge platform metrics define --file metric-definitions.json --env myorg

# Read one metric's recent trend
forge platform metrics trend scheduling_success --days 14 --env myorg

The Python CLI still registers metrics freshness and metrics evaluate, but the current Platform API has no dedicated metric-freshness or generic metric-evaluate route. Do not use those commands. The embedded production-eval operation is separate: it runs active eval definitions and persists their verdicts.

Coverage-Optimized Simulation (forge simulation)

In addition to the forge platform sim and forge platform simulation groups shared with the Go build, the Python CLI has a top-level forge simulation group that provides coverage-optimized simulation testing against context graphs. It automatically steers simulated conversations toward unvisited states, behaviors, and tools to maximize test coverage.

Each simulation turn follows a scoring loop:

1. The platform generates recommended user responses (graph-unaware)
2. An LLM classifier predicts which state each response would transition to
3. A scorer ranks responses by expected coverage value using graph structure
4. The highest-scoring response is sent as the simulated user message
5. Coverage state is updated based on the agent's response

Command	Description
forge simulation run	Execute a simulation with configurable sessions, turn budgets, and coverage targets
forge simulation plan	Generate a target spec from a natural-language objective (e.g., "test the cancellation flow end-to-end")
forge simulation bridge	Generate scenario variations from a natural-language objective, run multi-turn conversations with LLM-driven personas, and track coverage using interaction insights
forge simulation evaluate	Compare metric scores across simulation runs, including before/after diff mode
forge simulation cleanup	Delete ephemeral test users created by simulation runs

Simulations are highly configurable:

Setting	Default	Description
Sessions	3	Number of parallel conversations
Max turns	20	Maximum turns per session
Budget	100	Total turn budget across all sessions
Algorithm	frontier	Scoring algorithm: frontier, heatmap, or random
Temperament	random	Simulated user personality: cooperative, neutral, frustrated, confused, skeptical, or random

The forge simulation bridge command combines scenario generation with multi-turn conversation execution. You describe what you want to test in natural language, and the bridge generates diverse scenario variations, runs each as a full conversation with an LLM-driven persona, and collects interaction insights after every turn for coverage tracking.

# Generate and run 5 scenarios testing cancellation handling
forge simulation bridge --service "Scheduling" --objective "test cancellation edge cases" --scenarios 5 --env staging

Simulation bridge results are persisted locally across runs, enabling trend analysis and regression detection. Tag scenarios for selective execution (for example, forge simulation bridge --tag scheduling) to build a reusable test library that grows over time.

Streaming text smoke test (forge platform conversation text-ws-smoke)

The legacy Python command opens the Platform Sessions WebSocket, sends one message, and waits for the agent response. It requires an entity ID for context and accepts service, message, and conversation options. This command is not yet implemented in the Go binary; use it only when the legacy Python CLI is already part of your workflow.

Changelog (forge changelog)

The forge changelog show command provides cross-entity change traceability - tracking what changed across agents, context graphs, behaviors, and metrics over time. This gives teams visibility into configuration drift without relying on external version control tooling.

CLI Updates

The Python build updates itself from its git checkout rather than downloading a binary. It checks origin/main in the background (throttled) and, when updates are available, prompts before applying them. You can also update manually:

forge update

When updates are applied, uncommitted local changes are stashed during the update and restored afterward. If dependencies changed, poetry install is re-run automatically and the original command is re-executed.

Coding Agent Skills

The amigo-forge plugin is a public skill marketplace for Codex and Claude Code. It lets your coding agent drive the forge CLI for you. Instead of remembering command syntax, you describe what you want in plain language - "scope this agent before I build it", "validate my entity JSON", "run regression sims before I promote" - and the agent picks the matching skill and runs it against the forge binary you already have installed. The skills drive your local binary; they do not ship or install it.

The marketplace lives in the public amigo-ai-solutions/forge-skills repository and requires forge version 0.1.23 or newer on your PATH.

Install

Codex:

# Add the marketplace (one time)
codex plugin marketplace add amigo-ai-solutions/forge-skills

Then open Codex, run /plugins, choose the Amigo Forge marketplace, and install the forge plugin.

Claude Code:

# Add the marketplace (one time)
claude plugin marketplace add amigo-ai-solutions/forge-skills

# Install the forge plugin
claude plugin install forge@amigo-forge

List installed skills and plugins with /plugins in Codex or /plugin in Claude Code.

Available Skills

Skill	Use it to
forge-agent-design	Scope an agent before building - decide where each piece of complexity belongs (a deterministic function, a context graph state, an isolated skill, a router, or multiple agents). Start here.
forge-build-agent	Stand up the entities end to end: functions, then a context graph, the agent, skills, a service, and a pinned version set.
forge-validate	Run the local, no-auth pre-push validation gate over your entity JSON.
forge-sync	Read, edit, validate, and deploy entity data through forge platform commands - pull with get, then push with a dry run first, then apply.
forge-simulate	Regression-test and prove parity before promoting a version set, keeping a rollback path.

In any Forge project, describe what you want and the coding agent selects the matching skill. You can also invoke a skill explicitly:

# Codex
$forge-agent-design

# Claude Code
/forge:forge-agent-design

Keeping Skills Up to Date

Codex loads new skill versions after the marketplace catalog is refreshed and the installed plugin is updated. Refresh the marketplace catalog from a shell:

codex plugin marketplace upgrade amigo-forge

Then open /plugins, update the installed forge plugin if prompted, and start a new thread so Codex reloads the plugin instructions.

Claude Code auto-update is off by default for third-party marketplaces. Turn it on from /plugin > Marketplaces > amigo-forge > Enable auto-update so Claude Code refreshes the catalog and updates the plugin at startup. To update by hand, refresh the catalog and then update the plugin:

# Refresh the marketplace catalog
claude plugin marketplace update amigo-forge

# Update the installed plugin to the catalog's latest version
claude plugin update forge@amigo-forge

Run /reload-plugins afterward to activate the new version in the current session without restarting.

Typical Workflow

1. Pull current configurations from the platform to your local environment.
2. Make changes to the JSON configuration files.
3. Push to staging and run your test sets to validate.
4. Review results and iterate if tests fail.
5. Promote to production after validation passes.

This workflow supports both manual changes and automated optimization. Teams can use Agent Forge directly for planned configuration updates, or set up automated pipelines that use Agent Forge to deploy and test changes as part of a continuous improvement process.

When to Use Agent Forge

• Managing configurations across environments: Keep staging and production in sync with a controlled promotion process.
• Bulk updates: Modify multiple agents, behaviors, or evaluation criteria in a single operation.
• Scripted deployments: Integrate Agent Forge into CI/CD pipelines for automated testing and deployment.
• Audit and rollback: Maintain a complete history of configuration changes with the ability to revert.
• Building agents from scratch: Use Platform API commands to create agents, context graphs, and services entirely from the CLI.
• Coverage testing: Run simulation tests that automatically explore unvisited states and edge cases.

Use the Platform API developer guide for setup, authentication, and workspace configuration details. This reference page covers the Agent Forge command surface.