> For the complete documentation index, see [llms.txt](https://docs.amigo.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.amigo.ai/reference/agent-forge.md). # Agent Forge CLI Agent Forge is the CLI tool for managing agent configurations on the Amigo platform. It lets you create, update, version, and promote agent components programmatically rather than through the web interface. Agent Forge treats agent configurations as code. You sync configurations to local JSON files, make changes, and push them back to the platform. This gives you version control, reproducibility, and the ability to script deployment workflows. ## Choosing a Build Agent Forge ships as two builds that share the same `forge` command and most of the same command surface: * **Go CLI (current, recommended)** - A single self-contained binary with no runtime dependencies. This is the source of truth for the `forge` CLI going forward. New workspaces and new automation should use this build. **The Go CLI officially supports only the Amigo** [**Platform API**](https://docs.amigo.ai/developer-guide/platform-api/platform-api) - it drives all remote operations through the `forge platform ...` command tree. It does not officially support the Classic API (the legacy backend); use the Python CLI for Classic API workflows. * **Python CLI (legacy)** - The original Poetry-managed Python tool. It supports both the Platform API and the Classic API, and it carries a superset of commands - a handful of analytics, quality, and reporting commands have not yet been ported to the Go binary - but it is being retired for external use and now mainly serves internal Amigo workflows. Use it if you need the Classic API or one of the legacy-only commands listed in its tab below. Both builds understand the same entity model, the same `--env` flag, and the same `.env.platform.` configuration for the Platform API. The tabs below document each build; unless a section says otherwise, the Platform API command syntax is identical across builds. To tell which build you have, run `forge version` (Go prints a stamped binary version; the Python tool runs from a git checkout). ## What Agent Forge Manages Across its two builds, Agent Forge manages the following configuration and testing resources. Coverage differs by build: the current Go CLI targets Platform API resources, while the legacy Python CLI retains Classic API workflows and legacy-only analytics commands. The tabs below identify the supported command surface for each build. * **Agents**: Persona, background, directives, and communication style * **Context graphs**: Problem structure, states, transitions, and safety boundaries * **Metrics**: Evaluation criteria, scoring rubrics, and custom metric definitions * **Personas**: Synthetic user profiles for simulation testing (the primary way to manage personas) * **Scenarios**: Test situations for simulation testing * **Services**: Link an agent and context graph into a deployable unit * **Agent definitions**: Native (bring-your-own-framework) definitions for supported SDK frameworks (current Go CLI) * **Tools**: Versioned code packages (called Actions in the conceptual docs) * **Unit test sets**: Groups of tests with success criteria * **Unit tests**: Individual test cases * **User dimensions**: Attributes that segment users for evaluation and analysis {% tabs %} {% tab title="Go CLI (current)" %} The Go build is the recommended, go-forward CLI. It ships as a single binary and covers the full agent-building and platform-management workflow. {% hint style="info" %} The Go CLI officially supports only the Amigo **Platform API**. All remote operations go through the `forge platform ...` command tree. Classic API (legacy backend) workflows are not officially supported in this build - use the Python CLI tab for those. {% endhint %} ### Installation Agent Forge ships as a single binary with no runtime dependencies. On macOS, the installer detects the architecture, downloads the correct binary, verifies the SHA256 checksum, and places it on your PATH. ```bash curl -fsSL https://forge.platform.amigo.ai/install.sh | sh ``` Pre-built targets cover macOS (Intel and Apple Silicon) and Linux (amd64 and arm64). The public installer and the `forge update` self-updater currently target macOS. For Linux, obtain the binary through the release channel provided to your organization. After installation, configure credentials for your workspace: ```bash # Create environment file cp .env.platform.example .env.platform. # Edit with your Platform API URL, workspace ID, and API key or identity URL # Verify forge auth status --platform --env ``` You can also manage credentials as reusable profiles instead of environment files with the `forge platform config` command group (`add`, `use`, `list`, `show`, `doctor`, `remove`, `import-env`, `path`). ### Authentication The Go CLI authenticates against the Platform API. Pass the `--platform` flag on the auth commands to target the Platform identity service. Two methods are supported, selected automatically based on the environment configuration: device code login through the identity service, and static API keys. #### Device Code Login (Recommended) Device code authentication follows RFC 8628. When you run `forge auth login --platform`, the CLI requests a device code from the identity service, displays a short user code, opens your browser to an approval page, and polls for authorization. You verify that the code shown in the browser matches the code in your terminal and approve the request. These flows work in headless environments, SSH sessions, and CI pipelines where a browser cannot be opened inline. The approval page enforces that your browser session is scoped to the same workspace the device code targets - if your session is scoped to a different workspace, the page redirects you to workspace selection first, and after choosing the correct workspace you are returned to the approval page automatically. If you are not signed in at all, the sign-in flow preserves the approval page as the return destination through authentication and workspace selection, so you land back on the approval page without needing to re-open the CLI link. A session scoped to the wrong workspace or a session without any workspace selected cannot approve the code. The platform identity device code flow is workspace-scoped end to end. When you initiate a login, Forge sends the configured workspace ID along with the device code request, and the identity service binds the code to that workspace - the approver in the browser must hold a session scoped to the same workspace, and the resulting CLI token is scoped to it. This workspace enforcement applies at both the approval step and the token exchange step, ensuring that credentials are always tied to the intended workspace and preventing cross-workspace token misuse. Once approved, Forge receives an access token and refresh token automatically - no manual token management required. Platform identity tokens are stored in a workspace-keyed file cache under the operating system's user configuration directory; Forge creates the cache directory with owner-only access and writes token files with mode `0600`. An expired access token is refreshed with the stored refresh token without requiring re-authentication when refresh succeeds. Device code login replaces the need for a static API key for interactive CLI use. #### API Key Static bearer token authentication. Generate an API key from Amigo Console under **Developer > API Keys** and add it to your environment file. Platform API keys expire after a configured 1-90 days; Forge defaults new keys to 30 days. Rotate CI/CD credentials before expiry and delete keys that are no longer needed. #### Environment Configuration Platform API authentication reads from `.env.platform.` (preferred) or falls back to `.env.`. The following variables control the platform auth path: | Variable | Required | Description | | ----------------------- | ------------ | --------------------------------------------------------- | | `PLATFORM_API_URL` | Yes | Platform API URL for the target environment | | `PLATFORM_WORKSPACE_ID` | Yes | Workspace to authenticate against | | `PLATFORM_API_KEY` | One of these | Static API key (no login required) | | `IDENTITY_URL` | One of these | Platform identity service URL (enables device code login) | If `PLATFORM_API_KEY` is set, Forge uses it as a static bearer token. If `IDENTITY_URL` is set instead, Forge uses the device code flow via `forge auth login --platform`. Forge-native configuration fields are automatically translated to platform-native equivalents at deployment time. For example, audio filler phrases defined in Forge tool specs are converted to the platform's progress hint format, so agents configured through Forge work without manual migration. #### Auth Commands ```bash # Platform API login (platform identity device code) forge auth login --platform -e myorg # Check auth status forge auth status --platform -e myorg # Clear cached credentials forge auth logout --platform -e myorg ``` Always pass `--platform` on the Go CLI. The `--platform` flag is available on `login`, `logout`, and `status`. Without it, the auth commands operate on Classic API (legacy backend) credentials, which the Go build does not officially support. ### Command Groups The sections below cover the Agent Forge command surface, one command group at a time. Most commands support `--json` for structured output, and `--env` selects the target environment, enabling integration with scripts and CI/CD pipelines. (`forge validate`, which runs against local files, accepts `--env` but not `--json`.) #### Sync and Deployment **Pull from the platform** - Read entities down from the Platform API with the per-resource `get` and `list` commands, then edit them as JSON on disk: ```bash # List and fetch entities from the Platform API forge platform agent list --env myorg forge platform agent get --env myorg forge platform context-graph get --env myorg forge platform service get --env myorg ``` **Push to the platform** - Push local changes back with `forge platform push` (see **Bulk Push** below). Before applying changes, Agent Forge shows exactly what will be modified so you can review before confirming. {% hint style="info" %} The Go binary still carries the older `sync-to-local` / `sync-to-remote` commands, but they operate against the Classic API (legacy backend), which the Go build does not officially support. For Platform API work, pull with `forge platform get` / `list` and push with `forge platform push`. For Classic API sync, use the Python CLI. {% endhint %} **Pre-Push Validation** - Agent Forge validates context graphs before pushing to the platform and surfaces warnings for common authoring mistakes. Validation runs automatically during `forge platform push`, and you can run it on local files at any time with `forge validate` (no auth required). The canonical value lint detects phone numbers, email addresses, and URLs hardcoded into context graph state prose. Inline canonical values cause silent data drift - when graphs are cloned or updated, hardcoded digits can be accidentally mutated, and the agent reads incorrect information to callers. The validator scans prose fields in every state (descriptions, instructions, boundary constraints, exit conditions, and action descriptions) and emits a warning for each match, identifying the state, field, and value. It catches phone numbers in digit form (e.g., `555-010-1234`), phone numbers in spelled-out TTS form (e.g., "five five five zero one zero..."), email addresses, and URLs. To fix a warning, move the canonical value into structured context - such as a location entity in the world model or a workspace setting - and reference it abstractly in the state prose. **Environment Support** - Agent Forge supports separate staging and production environments. Changes are deployed to staging first, validated through testing, and then promoted to production. Platform push operates on agents, context graphs, and services, laid out per environment: ``` project/ local/ staging/ entity_data/ agent/ context_graph/ service/ .platform_id_map.json production/ entity_data/ (same structure) .platform_id_map.json ``` **Bulk Push** - Push local entity configurations to the Platform API in a single operation: ```bash forge platform push --all --env myorg --apply ``` Supports selective push by entity type (`-e agent`, `-e context-graph`, `-e service`). The Go build maintains a local-to-platform UUID map (`local//.platform_id_map.json`) so repeated pushes update the same platform entities. #### Agent Building The `forge platform` command group provides broad CLI coverage for common Platform API workflows, enabling agent building and workspace management without the web interface. Build a complete agent from the CLI in four steps: 1. **Create agent** and agent version with identity, background, and behaviors 2. **Create context graph** and version with states, transitions, and exit conditions 3. **Create service** linking the agent and context graph together 4. **Add skills** (optional) for LLM-backed micro-agent capabilities ```bash forge platform agent create --name "My Agent" --env myorg forge platform agent create-version --file agent-version.json --env myorg forge platform context-graph create --name "My Context Graph" --env myorg forge platform context-graph create-version --file context-graph-version.json --env myorg forge platform service create --name "My Service" --agent-id --context-graph-id --env myorg ``` CLI support is organized into these resource groups: | Resource Group | Commands | | --------------------- | -------------------------------------------------------------------------------------------- | | **Core** | `workspace`, `agent`, `agent-definition`, `context-graph`, `service`, `version-set`, `skill` | | **Voice & Text** | `conversation`, `run`, `voice-settings`, `voice-check` | | **Data** | `integration`, `external-integration`, `function`, `fhir`, `workspace-table`, `data-query` | | **Surfaces** | `surface` | | **Testing** | `simulation`, `sim`, `tool-test`, `regression`, `agent-run` | | **Access & Identity** | `api-key`, `role-grant`, `role-assignment`, `external-role`, `use-case` | | **Operations** | `config`, `audit`, `session`, `triggers` | #### Framework Agent Definitions and Runs Forge supports customer-authored agents built for the Anthropic Claude Agent SDK (`claude-agent-sdk`) and OpenAI Agents SDK (`openai-agents`). Definitions are immutable and versioned after registration. | Command | Description | | --------------------------------------------------------- | ----------------------------------------------------- | | `forge platform agent-definition validate` | Validate a definition without storing it | | `forge platform agent-definition register` | Register a new definition or version an existing name | | `forge platform agent-definition list` | List active definitions across all result pages | | `forge platform agent-definition get ` | Retrieve a definition and its version metadata | | `forge platform agent-definition archive ` | Soft-archive a definition after confirmation | `validate` and `register` require `--name` and exactly one JSON source: `--file` or `--body`. Re-registering an identical body reports it as unchanged; a changed body creates a new immutable version. `forge platform agent-run create` dispatches either a service-backed framework run or a native definition run. Provide exactly one run source: ```bash # Platform-authored service configuration forge platform agent-run create \ --service-id \ --framework claude-agent-sdk \ --message "Review this case" \ --wait \ --env myorg # Registered native definition (latest version unless pinned) forge platform agent-run create \ --definition-id \ --message "Review this case" \ --wait \ --env myorg # Inline native definition for development forge platform agent-run create \ --native-config definition.json \ --message "Review this case" \ --wait \ --env myorg ``` Without `--wait`, create returns the run ID immediately. Use `agent-run get --run-id ` for a later snapshot, or `agent-run harness-context --service-id ` to inspect the framework-neutral context exposed by a service. See [Agent Definitions](https://docs.amigo.ai/developer-guide/platform-api/functions/agent-definitions) in the developer guide for definition shapes and safety constraints. #### Voice Configuration Presets `forge platform service voice-config` can read the current configuration, apply an inline JSON body, or apply a named preset. The CLI recognizes `ultra_low_latency`, `balanced`, `quality`, `gpt_realtime`, and `gpt_live`. The `gpt_realtime` preset selects OpenAI Realtime speech-to-speech; provider, model, and regional availability are still enforced by the platform. ```bash forge platform service voice-config --get --env myorg forge platform service voice-config --preset gpt_realtime --env myorg ``` #### Simulation Use `forge platform simulation` for the supported branch-and-bound coverage surface. `forge platform sim bridge` remains a shortcut to the same bridge workflow, and `forge platform sim config-to-policy` maps a configuration object to turn-policy fields. Legacy configuration-space commands such as `sim create`, `sample`, `evaluate`, `status`, `points`, `summary`, and `complete` target retired endpoints and should not be used. **Simulation Coverage** The `forge platform simulation` command group manages branch-and-bound simulation coverage runs that systematically explore context graph state space. | Command | Description | | ------------------------------------------ | ------------------------------------------------------------------------------- | | `forge platform simulation run create` | Create a new coverage run for a service | | `forge platform simulation run list` | List coverage runs for a service | | `forge platform simulation run complete` | Complete a run | | `forge platform simulation session create` | Create a session within a coverage run | | `forge platform simulation session step` | Step a session forward with a simulated user message | | `forge platform simulation session fork` | Fork a session into children at a decision point, each with a different message | | `forge platform simulation session score` | Score a session against configured metrics | | `forge platform simulation graph show` | Retrieve the coverage knowledge graph with topology overlay and ghost nodes | | `forge platform simulation graph paths` | List observed paths through the coverage graph | See [Simulation Coverage](/testing/testing/simulations.md#simulation-coverage) for conceptual background. **Simulation Caller and Entity Context** `forge platform simulation session create` accepts `--caller-id` for a simulated caller phone number in E.164 format and `--entity-id` for direct binding to a known world entity. `forge platform sim bridge` accepts `--entity-id` but not `--caller-id`. Direct entity context is useful for regression tests against a known patient or account fixture. Omit the context flags to simulate an unknown caller. ```bash # Bridge scenarios against a known entity forge platform sim bridge --service-id -o "Test known entity flow" --entity-id --env myorg # Branch-and-bound coverage session with entity context forge platform simulation session create --run-id --service-id --entity-id --env myorg ``` **Tracked Platform Simulation Sessions** The `forge platform sim` group also drives tracked platform simulation sessions used by the Agent Readiness workflow: | Command | Description | | ----------------------------------- | --------------------------------------------------------------------------- | | `forge platform sim smoke-test` | Single-turn sanity check via a tracked platform session | | `forge platform sim session-create` | Create a tracked simulation session (accepts `--caller-id` / `--entity-id`) | | `forge platform sim run-create` | Create a tracked simulation run | | `forge platform sim run-list` | List simulation runs with filtering | | `forge platform sim run-complete` | Mark a simulation run as complete | #### Text Conversation Testing The `forge platform conversation` command group tests text conversations through the REST API without a phone or browser. These commands are useful during initial setup, after configuration changes, or as part of a deployment validation pipeline. | Command | Description | | ------------------------------------------ | ----------------------------------------------------------------------------------------- | | `forge platform conversation list` | List conversation runs through the unified Runs view; use `run list` for new automation | | `forge platform conversation get` | Retrieve conversation metadata, transcript, and optional tool calls | | `forge platform conversation create` | Create a durable text conversation for a service and optional entity context | | `forge platform conversation send-message` | Send a message, wait for the final reply when possible, and optionally evaluate it inline | | `forge platform conversation poll` | Claim a completed background reply after a pending send | | `forge platform conversation close` | Close a text thread so the next inbound message starts a new conversation | Create a durable text conversation, then send user messages through the REST turns endpoint and display the agent's response: ```bash # Create the conversation CONV_ID=$(forge platform conversation create \ --service-id \ --env myorg \ --json | jq -r .id) # Send the first message forge platform conversation send-message \ --conversation-id "$CONV_ID" \ --message "What appointments are available tomorrow?" \ --env myorg # Continue the same conversation forge platform conversation send-message \ --conversation-id "$CONV_ID" \ --message "How about 2pm?" \ --env myorg # With patient context forge platform conversation create --service-id --entity-id --env myorg ``` `send-message` requires an existing conversation ID. Use `conversation create` first and pass the returned ID on subsequent calls to continue the same conversation thread. **Inline Reply Evaluation** `send-message` can evaluate a final reply in the same command. Repeat `--must-contain` or `--must-not-contain` for fast local checks. Repeat `--judge-criteria` for model-based checks against natural-language requirements; judge checks require configured judge credentials and `--i-understand-pii` because the reply may contain sensitive data. ```bash forge platform conversation send-message \ --conversation-id "$CONV_ID" \ --message "I need to move tomorrow's appointment" \ --must-contain "appointment" \ --must-not-contain "I cannot help" \ --judge-criteria "The reply offers a concrete next step" \ --i-understand-pii \ --env myorg ``` Forge requests a final, filler-free reply. If background work is still pending, it does not grade the interim acknowledgement; use `conversation poll` to claim the completed reply. The Go binary still registers `forge platform conversation text-ws-smoke` as a deferred placeholder and does not run the WebSocket test. Use the REST `send-message` flow above, connect through the documented [Sessions WebSocket](https://docs.amigo.ai/developer-guide/platform-api/platform-api/sessions) directly, or use the legacy Python command described in its tab. #### Unified Runs `forge platform run` is the canonical read surface across conversation runs and framework runs. It uses the same channel-neutral run identifiers as the Developer Console Runs page. | Command | Description | | ---------------------------- | ----------------------------------------------------------------------------------- | | `forge platform run list` | List runs with kind, channel, and status filters plus continuation-token pagination | | `forge platform run summary` | Show aggregate totals, live counts, status counts, and kind counts | | `forge platform run get` | Retrieve one run by its channel-neutral run ID | ```bash forge platform run list --kind conversation --channel voice --status live --env myorg forge platform run summary --kind conversation --env myorg forge platform run get --env myorg ``` Use `conversation` for conversation-specific create, send, poll, and close operations. Use `agent-run` to dispatch framework runs. Use `run` when you need a single inventory across both families. See [Runs](https://docs.amigo.ai/developer-guide/platform-api/conversations/runs) in the developer guide. #### Tool Testing The `forge platform tool-test` commands let you test context graph tools without making phone calls: | Command | Description | | ---------------------------------- | --------------------------------------------------------------- | | `forge platform tool-test resolve` | List available tools for a service with input schemas | | `forge platform tool-test execute` | Execute a tool with custom parameters and optional dry run mode | #### Trigger Management The `forge platform triggers` command group manages action automations that run on a schedule, in response to supported platform events, or through a manual fire. | Command | Description | | -------------------------------- | ---------------------------------------------------------------------------- | | `forge platform triggers create` | Create a trigger with an action binding and schedule or supported event type | | `forge platform triggers list` | List triggers with active/inactive filtering | | `forge platform triggers get` | Get trigger details including next fire time | | `forge platform triggers update` | Update trigger configuration | | `forge platform triggers delete` | Delete a trigger | | `forge platform triggers pause` | Pause a trigger's schedule | | `forge platform triggers resume` | Resume a paused trigger | | `forge platform triggers fire` | Manually fire a trigger for testing | | `forge platform triggers runs` | View trigger execution history | Scheduled triggers use cron expressions. Event-based triggers match a supported world-model event and can narrow matches with an event filter. Manual fires use the same durable execution path, which makes them useful for testing a trigger before activation. See [Outbound](/channels/outbound.md) for how triggers fit into automated contact patterns. #### Platform Functions The `forge platform function` command group manages platform functions - declarative SQL, Python, AI, and table-valued (UDTF) functions that agents can call mid-conversation. Table-valued functions return rows rather than a single value. | Command | Description | | ---------------------------------- | ------------------------------------------------------------------ | | `forge platform function register` | Register a new platform function with its definition and metadata | | `forge platform function list` | List all registered functions in the workspace | | `forge platform function test` | Execute a function with test parameters and inspect the result | | `forge platform function delete` | Remove a function registration | | `forge platform function query` | Run an open-scope SQL query against workspace data | | `forge platform function catalog` | Display the full function catalog with signatures and descriptions | | `forge platform function sync` | Sync function definitions between local files and the platform | See [Platform Functions](/agent/platform-functions.md) for conceptual background. ### CLI Updates The Go binary ships a built-in self-update command (macOS) that upgrades the binary in place: ```bash forge update # download and install the latest release forge update --check # report whether an update is available; install nothing forge update --version 1.4.2 # pin an exact release (up- or down-grade) ``` Here `forge update` resolves the target version, downloads the release for your Mac's architecture from the same Forge CDN the installer uses, verifies its SHA256 checksum, sanity-checks the downloaded binary before swapping it, and atomically replaces the running binary in place. The download always comes from the immutable, version-pinned release path, so a publish in progress can never mix generations mid-download. | Flag | Purpose | | ------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | | `--check` | Report whether an update is available and exit without downloading or writing anything. | | `--version ` | Install this exact release instead of the latest, up- or down-grade. A downgrade below the installed version prompts for confirmation first. | | `--force` | Reinstall even when already up to date, overwrite a locally built `dev` binary, and skip the downgrade confirmation prompt. | | `--yes`, `-y` | Skip the downgrade confirmation prompt for non-interactive or scripted use. | If the install directory is not writable (for example, `forge` lives in `/usr/local/bin`), `update` fails with guidance to re-run as `sudo forge update` rather than silently escalating privileges. On Linux, reinstall from your organization's release channel to upgrade. {% endtab %} {% tab title="Python CLI (legacy)" %} {% hint style="warning" %} The Python CLI is the legacy build and is being retired. Prefer the **Go CLI** (see the other tab) for all new work. Use the Python build only if you depend on one of the legacy-only commands listed below that has not yet been ported to the Go binary. {% endhint %} The Python CLI shares the Platform API command surface with the Go build - `auth`, `validate`, `forge platform push`, and the `forge platform` resource groups for agents, context graphs, services, skills, functions, conversations, tool tests, simulation, surfaces, and integrations. For those, follow the **Go CLI** tab; the syntax is the same. In addition, because the Python build also supports the Classic API, it offers the legacy `sync-to-local` / `sync-to-remote` workflow against the legacy backend. This tab documents what is different in the Python build. ### Installation The Python CLI is not a standalone binary - it runs from a git checkout with a full Python runtime, managed by Poetry. 1. Install Python 3.13 (the repo pins `3.13.5`, e.g. via `pyenv install 3.13.5`). 2. Install [Poetry](https://python-poetry.org/) 2.x. 3. From the repo root, install dependencies and activate the environment: ```bash poetry install $(poetry env activate) ``` Commands run as `poetry run forge ...` (or just `forge ...` inside the activated environment). Because the tool runs from a checkout, it must be invoked from the repository root. ### Authentication Unlike the Go build, the Python CLI supports **both** the Classic API (legacy backend) and the Platform API. The `--platform` flag selects which surface a command targets - omit it for the Classic API, pass it for the Platform API. Both use device code login and static API keys. Configuration is read from `.env.` (Classic API) and `.env.platform.` (Platform API) files. For the Platform-API authentication details (device code flow, environment variables), see the **Go CLI** tab. ### Command-name differences The command groups are largely identical to the Go build, with a few naming differences to watch for: * The trigger group is `forge platform trigger` (singular) in the Python CLI, versus `forge platform triggers` (plural) in the Go CLI. ### Legacy-only commands The following commands are implemented in the Python build only. They are recognized but not yet functional in the Go binary, so if you rely on any of them, use the Python CLI until they are ported. #### Analytics (`forge analyze`) The `forge analyze` command group provides SQL-based exploration of workspace data directly from the CLI, replacing the need for external analytics tools. | Command | Description | | ------------------------ | ------------------------------------------------------------------------------------------------------------- | | `forge analyze query` | Execute ad-hoc SQL SELECT queries (inline or from file). Results are capped and queries are time-bounded. | | `forge analyze describe` | Preview a query's output schema without executing it - useful for validating JOINs and checking column types. | | `forge analyze tables` | List available tables in the workspace schema. Supports SQL LIKE patterns for filtering. | | `forge analyze schema` | Describe a table's columns: names, data types, and comments. | | `forge analyze sample` | Preview sample rows from a table (default 5, max 20). | | `forge analyze detail` | Rich table metadata: row count, size, partitioning, column nullability, data freshness. | | `forge analyze profile` | Profile a column's data distribution: cardinality, null rate, min/max values. | | `forge analyze catalog` | Display the full data catalog reference offline without a database connection. | Pre-built analytics query templates cover common patterns like conversation volume, tool performance, and metric trends: ```bash # List templates forge analyze template list # Run a template with parameters forge analyze template run conversation-volume -P days=7 ``` #### Insights (`forge platform insights`) The `forge platform insights` command group provides conversational data exploration from the CLI, wrapping the platform's [Insights Agent](/intelligence-and-analytics/intelligence/analytics-dashboards.md#insights-agent) capabilities - workspace queries, schema metadata, and health digests. | Command | Description | | ------------------------------------- | ------------------------------------------------------------------------------------------------------------ | | `forge platform insights sql` | Execute a SQL query against workspace data and return formatted results | | `forge platform insights schema` | Describe available tables, columns, and functions in the workspace schema | | `forge platform insights digest` | Generate an AI-powered digest summarizing recent workspace activity, entity counts, and data quality signals | | `forge platform insights suggestions` | Get suggested starter questions based on the workspace's data and recent activity | ```bash # Execute a SQL query against the workspace data warehouse forge platform insights sql "SELECT ..." --env myorg # Read SQL from a file forge platform insights sql --sql-file my_query.sql --env myorg --json # List available tables, columns, and functions forge platform insights schema --env myorg # Get a workspace health digest with entity counts and data quality signals forge platform insights digest --env myorg # Get suggested starter questions for exploring workspace data forge platform insights suggestions --env myorg ``` #### Call Trace Analysis (`forge platform trace`) The `forge platform trace` command group provides call trace analysis from the CLI, wrapping the platform's [trace analysis](/intelligence-and-analytics/intelligence/call-intelligence.md#call-trace-analysis) capabilities - deep call understanding from the intelligence pipeline. | Command | Description | | --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | | `forge platform trace list` | List call traces with filters for date range, service, quality score, outcome, and direction | | `forge platform trace get` | Get detailed trace analysis for a specific call, including emotional arc, decision moments, component attribution, and coaching recommendations | ```bash # List recent trace analyses forge platform trace list --env myorg # Filter by outcome and lookback window forge platform trace list --outcome failed --days 7 --env myorg # Get detailed trace analysis for a specific call forge platform trace get --env myorg ``` Trace analysis provides: * **Emotional arc** - How caller sentiment evolved across the conversation * **Key decision moments** - Critical points with quality assessment and causal attribution * **Coaching recommendations** - Actionable improvements tied to specific call moments * **Counterfactuals** - Alternative actions that could have changed the outcome * **Signal-response alignment** - Whether the agent responded appropriately to caller signals * **Interaction dynamics** - Turn-taking quality, rapport trajectory, and repair effectiveness Trace output uses rich formatting with colored outcome indicators and structured digest sections for quick scanning of call quality issues. #### Conversation Quality Check The `forge quality check` command scans workspace conversations for agent behavioral issues - stuck loops, degenerate output, repetition, and other quality problems. It queries production conversation data directly and runs pattern-based detectors to surface problematic interactions. ```bash # Scan last 24 hours forge quality check # Wider window with message snippets forge quality check --days 7 --verbose # Structured output for scripting forge quality check --json ``` | Detector | What It Finds | | -------------------------- | ------------------------------------------------------------------------------------------ | | **Character degeneration** | Repeated characters, low entropy output, stuttering patterns | | **Stuck agent loops** | Agent repeats the same response while the caller changes topics | | **Repetitive patterns** | High similarity across sliding message windows | | **Word salad** | Incoherent output patterns like or-chains and excessive word repetition | | **Phantom success** | Agent claims a tool call succeeded when the tool actually returned an error | | **Wrong tool inputs** | A tool is called with parameters that do not match what the caller actually asked for | | **Ungrounded claims** | The agent asserts capabilities or facts not supported by the configured entity definitions | | **Safety / PII** | The agent leaks sensitive information or provides unsafe guidance | Some detectors (phantom mismatch, wrong tool inputs, ungrounded claims, and safety/PII) require an LLM key to run. Results include conversation IDs, timestamps, detector names, and severity. Use `--verbose` to see the actual message excerpts that triggered each finding. See [Voice Simulation](/testing/testing/voice-simulation.md) and [Drift Detection](/testing/testing/drift-detection.md) for related quality monitoring capabilities. #### Metrics Management (`forge platform metrics`) The legacy Python CLI exposes workspace metric settings and current metric reads. Use only commands backed by the current Platform API: | Command | Description | | ------------------------------------------- | ---------------------------------------------------------- | | `forge platform metrics settings` | View built-in and custom workspace metric definitions | | `forge platform metrics define` | Replace the custom definitions supplied in a JSON document | | `forge platform metrics list` | List current metric values | | `forge platform metrics catalog` | List active built-in and custom catalog entries | | `forge platform metrics get ` | Read values for one metric | | `forge platform metrics trend ` | Read a metric time series | ```bash # View current metric settings forge platform metrics settings --env myorg # Define a metric forge platform metrics define --file metric-definitions.json --env myorg # Read one metric's recent trend forge platform metrics trend scheduling_success --days 14 --env myorg ``` The Python CLI still registers `metrics freshness` and `metrics evaluate`, but the current Platform API has no dedicated metric-freshness or generic metric-evaluate route. Do not use those commands. The embedded [production-eval operation](https://docs.amigo.ai/developer-guide/platform-api/safety/production-evals#evaluating-a-call) is separate: it runs active eval definitions and persists their verdicts. #### Coverage-Optimized Simulation (`forge simulation`) In addition to the `forge platform sim` and `forge platform simulation` groups shared with the Go build, the Python CLI has a top-level `forge simulation` group that provides coverage-optimized simulation testing against context graphs. It automatically steers simulated conversations toward unvisited states, behaviors, and tools to maximize test coverage. Each simulation turn follows a scoring loop: 1. The platform generates recommended user responses (graph-unaware) 2. An LLM classifier predicts which state each response would transition to 3. A scorer ranks responses by expected coverage value using graph structure 4. The highest-scoring response is sent as the simulated user message 5. Coverage state is updated based on the agent's response | Command | Description | | --------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `forge simulation run` | Execute a simulation with configurable sessions, turn budgets, and coverage targets | | `forge simulation plan` | Generate a target spec from a natural-language objective (e.g., "test the cancellation flow end-to-end") | | `forge simulation bridge` | Generate scenario variations from a natural-language objective, run multi-turn conversations with LLM-driven personas, and track coverage using interaction insights | | `forge simulation evaluate` | Compare metric scores across simulation runs, including before/after diff mode | | `forge simulation cleanup` | Delete ephemeral test users created by simulation runs | Simulations are highly configurable: | Setting | Default | Description | | --------------- | ---------- | -------------------------------------------------------------------------------------------------------- | | **Sessions** | 3 | Number of parallel conversations | | **Max turns** | 20 | Maximum turns per session | | **Budget** | 100 | Total turn budget across all sessions | | **Algorithm** | `frontier` | Scoring algorithm: `frontier`, `heatmap`, or `random` | | **Temperament** | `random` | Simulated user personality: `cooperative`, `neutral`, `frustrated`, `confused`, `skeptical`, or `random` | The `forge simulation bridge` command combines scenario generation with multi-turn conversation execution. You describe what you want to test in natural language, and the bridge generates diverse scenario variations, runs each as a full conversation with an LLM-driven persona, and collects interaction insights after every turn for coverage tracking. ```bash # Generate and run 5 scenarios testing cancellation handling forge simulation bridge --service "Scheduling" --objective "test cancellation edge cases" --scenarios 5 --env staging ``` Simulation bridge results are persisted locally across runs, enabling trend analysis and regression detection. Tag scenarios for selective execution (for example, `forge simulation bridge --tag scheduling`) to build a reusable test library that grows over time. #### Streaming text smoke test (`forge platform conversation text-ws-smoke`) The legacy Python command opens the Platform Sessions WebSocket, sends one message, and waits for the agent response. It requires an entity ID for context and accepts service, message, and conversation options. This command is not yet implemented in the Go binary; use it only when the legacy Python CLI is already part of your workflow. #### Changelog (`forge changelog`) The `forge changelog show` command provides cross-entity change traceability - tracking what changed across agents, context graphs, behaviors, and metrics over time. This gives teams visibility into configuration drift without relying on external version control tooling. ### CLI Updates The Python build updates itself from its git checkout rather than downloading a binary. It checks `origin/main` in the background (throttled) and, when updates are available, prompts before applying them. You can also update manually: ```bash forge update ``` When updates are applied, uncommitted local changes are stashed during the update and restored afterward. If dependencies changed, `poetry install` is re-run automatically and the original command is re-executed. {% endtab %} {% endtabs %} ## Coding Agent Skills The `amigo-forge` plugin is a public skill marketplace for Codex and [Claude Code](https://docs.claude.com/en/docs/claude-code/overview). It lets your coding agent drive the `forge` CLI for you. Instead of remembering command syntax, you describe what you want in plain language - "scope this agent before I build it", "validate my entity JSON", "run regression sims before I promote" - and the agent picks the matching skill and runs it against the `forge` binary you already have installed. The skills drive your local binary; they do not ship or install it. The marketplace lives in the public [`amigo-ai-solutions/forge-skills`](https://github.com/amigo-ai-solutions/forge-skills) repository and requires `forge` version 0.1.23 or newer on your PATH. ### Install Codex: ```bash # Add the marketplace (one time) codex plugin marketplace add amigo-ai-solutions/forge-skills ``` Then open Codex, run `/plugins`, choose the **Amigo Forge** marketplace, and install the `forge` plugin. Claude Code: ```bash # Add the marketplace (one time) claude plugin marketplace add amigo-ai-solutions/forge-skills # Install the forge plugin claude plugin install forge@amigo-forge ``` List installed skills and plugins with `/plugins` in Codex or `/plugin` in Claude Code. ### Available Skills | Skill | Use it to | | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `forge-agent-design` | Scope an agent before building - decide where each piece of complexity belongs (a deterministic function, a context graph state, an isolated skill, a router, or multiple agents). Start here. | | `forge-build-agent` | Stand up the entities end to end: functions, then a context graph, the agent, skills, a service, and a pinned version set. | | `forge-validate` | Run the local, no-auth pre-push validation gate over your entity JSON. | | `forge-sync` | Read, edit, validate, and deploy entity data through `forge platform` commands - pull with `get`, then push with a dry run first, then apply. | | `forge-simulate` | Regression-test and prove parity before promoting a version set, keeping a rollback path. | In any Forge project, describe what you want and the coding agent selects the matching skill. You can also invoke a skill explicitly: ``` # Codex $forge-agent-design # Claude Code /forge:forge-agent-design ``` ### Keeping Skills Up to Date Codex loads new skill versions after the marketplace catalog is refreshed and the installed plugin is updated. Refresh the marketplace catalog from a shell: ```bash codex plugin marketplace upgrade amigo-forge ``` Then open `/plugins`, update the installed `forge` plugin if prompted, and start a new thread so Codex reloads the plugin instructions. Claude Code auto-update is off by default for third-party marketplaces. Turn it on from `/plugin` > **Marketplaces** > `amigo-forge` > **Enable auto-update** so Claude Code refreshes the catalog and updates the plugin at startup. To update by hand, refresh the catalog and then update the plugin: ```bash # Refresh the marketplace catalog claude plugin marketplace update amigo-forge # Update the installed plugin to the catalog's latest version claude plugin update forge@amigo-forge ``` Run `/reload-plugins` afterward to activate the new version in the current session without restarting. ## Typical Workflow 1. **Pull current configurations** from the platform to your local environment. 2. **Make changes** to the JSON configuration files. 3. **Push to staging** and run your test sets to validate. 4. **Review results** and iterate if tests fail. 5. **Promote to production** after validation passes. This workflow supports both manual changes and automated optimization. Teams can use Agent Forge directly for planned configuration updates, or set up automated pipelines that use Agent Forge to deploy and test changes as part of a continuous improvement process. ## When to Use Agent Forge * **Managing configurations across environments**: Keep staging and production in sync with a controlled promotion process. * **Bulk updates**: Modify multiple agents, behaviors, or evaluation criteria in a single operation. * **Scripted deployments**: Integrate Agent Forge into CI/CD pipelines for automated testing and deployment. * **Audit and rollback**: Maintain a complete history of configuration changes with the ability to revert. * **Building agents from scratch**: Use Platform API commands to create agents, context graphs, and services entirely from the CLI. * **Coverage testing**: Run simulation tests that automatically explore unvisited states and edge cases. {% hint style="info" %} Use the [Platform API developer guide](https://docs.amigo.ai/developer-guide/platform-api/platform-api) for setup, authentication, and workspace configuration details. This reference page covers the Agent Forge command surface. {% endhint %} --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.amigo.ai/reference/agent-forge.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.