# Metric Store

The metric store computes, stores, and serves metrics across all channels from a single config-driven pipeline. 41 built-in metrics ship with every workspace. Custom metrics can be added through the settings API without code changes.

## Metric Settings

Metric definitions are stored in workspace settings. The settings API lets you view all definitions (built-in and custom) and add or modify custom ones.

### Get Metric Definitions

```
GET /v1/{workspace_id}/settings/metrics
```

Returns all metric definitions for the workspace, including 41 built-in metrics and any custom definitions.

{% tabs %}
{% tab title="curl" %}

```bash
curl -H "Authorization: Bearer $API_KEY" \
  "https://api.platform.amigo.ai/v1/$WORKSPACE_ID/settings/metrics"
```

{% endtab %}

{% tab title="Python" %}

```python
resp = client.get(f"/v1/{workspace_id}/settings/metrics")
definitions = resp.json()["definitions"]

custom = [d for d in definitions if not d["builtin"]]
print(f"{len(custom)} custom metrics defined")
```

{% endtab %}

{% tab title="TypeScript" %}

```typescript
const { data } = await client.GET(
  "/v1/{workspace_id}/settings/metrics",
  { params: { path: { workspace_id } } }
);
```

{% endtab %}
{% endtabs %}

**Response:**

```json
{
  "definitions": [
    {
      "key": "voice_quality_score",
      "name": "Voice Quality Score",
      "metric_type": "numerical",
      "source": "call_intelligence",
      "extraction_mode": "static",
      "extract_path": "$.quality_score",
      "aggregation": "avg",
      "unit": "score",
      "active": true,
      "builtin": true
    }
  ]
}
```

### Create or Update Custom Metrics

```
PUT /v1/{workspace_id}/settings/metrics
```

Replaces the custom metric definitions for the workspace. Built-in metrics are preserved automatically - you only need to include your custom definitions.

{% tabs %}
{% tab title="curl" %}

```bash
curl -X PUT \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "definitions": [
      {
        "key": "empathy_score",
        "name": "Agent Empathy",
        "description": "How empathetic was the agent during the conversation",
        "metric_type": "numerical",
        "source": "call_intelligence",
        "event_types": ["call_intelligence.row"],
        "extraction_mode": "ai_query",
        "model_tier": "balanced",
        "ai_query_prompt": "Rate how empathetic the agent was on a scale of 1-10. Consider whether the agent acknowledged concerns, showed understanding, and responded with appropriate sensitivity. Output ONLY a single integer from 1 to 10.",
        "aggregation": "avg",
        "unit": "score",
        "valid_range_min": 1.0,
        "valid_range_max": 10.0
      }
    ]
  }' \
  "https://api.platform.amigo.ai/v1/$WORKSPACE_ID/settings/metrics"
```

{% endtab %}

{% tab title="Python" %}

```python
resp = client.put(
    f"/v1/{workspace_id}/settings/metrics",
    json={
        "definitions": [
            {
                "key": "empathy_score",
                "name": "Agent Empathy",
                "metric_type": "numerical",
                "source": "call_intelligence",
                "event_types": ["call_intelligence.row"],
                "extraction_mode": "ai_query",
                "model_tier": "balanced",
                "ai_query_prompt": "Rate empathy 1-10. Output ONLY the integer.",
                "aggregation": "avg",
                "unit": "score",
            }
        ]
    },
)
```

{% endtab %}
{% endtabs %}

**Permissions:** Admin or Owner role required.

### Metric Definition Fields

| Field                | Required    | Description                                                                                                     |
| -------------------- | ----------- | --------------------------------------------------------------------------------------------------------------- |
| `key`                | Yes         | Unique identifier. Lowercase alphanumeric and underscores (`^[a-z][a-z0-9_]*$`). Max 64 characters.             |
| `name`               | Yes         | Display name. Max 256 characters.                                                                               |
| `description`        | No          | Human-readable description. Max 2000 characters.                                                                |
| `metric_type`        | Yes         | `numerical`, `categorical`, or `boolean`                                                                        |
| `source`             | Yes         | Data source. See [Sources](#sources) below.                                                                     |
| `event_types`        | Yes         | Event types to scan. For `call_intelligence` source, use `["call_intelligence.row"]`.                           |
| `extraction_mode`    | Yes         | How to extract the value. See [Extraction Modes](#extraction-modes).                                            |
| `model_tier`         | No          | AI model tier: `free`, `fast`, `balanced`, `max`. Default: `free`. Required for `ai_query`.                     |
| `ai_query_prompt`    | Conditional | Prompt template for `ai_query` mode. Required when `extraction_mode` is `ai_query`. Max 8000 characters.        |
| `extract_path`       | Conditional | JSON path for `static` mode (e.g., `$.quality_score`).                                                          |
| `aggregation`        | No          | Aggregation function: `count`, `sum`, `avg`, `min`, `max`, `count_distinct`, `ratio`, `rate`. Default: `count`. |
| `unit`               | No          | Display unit (e.g., `score`, `%`, `seconds`). Max 32 characters.                                                |
| `period_granularity` | No          | `hourly` or `daily`. Default: `daily`.                                                                          |
| `channel_scope`      | No          | `all`, `voice`, `text`, `surface`, `inbound`, `outbound`. Default: `all`.                                       |
| `valid_range_min`    | No          | Minimum valid value for numerical metrics. Values below are dropped.                                            |
| `valid_range_max`    | No          | Maximum valid value for numerical metrics. Values above are dropped.                                            |
| `active`             | No          | Set to `false` to disable without deleting. Default: `true`.                                                    |

### Sources

| Source                | What It Reads                                                                                                |
| --------------------- | ------------------------------------------------------------------------------------------------------------ |
| `call_intelligence`   | Call and text conversation summaries. Use for evaluating conversation quality, outcomes, and agent behavior. |
| `world_events`        | All events across all domains. Use for event-level metrics.                                                  |
| `surface_events`      | Form submission lifecycle events.                                                                            |
| `emotion_events`      | Audio emotion analysis events.                                                                               |
| `connector_events`    | Data connector sync events.                                                                                  |
| `voice_judge_results` | Audio-native voice quality scores.                                                                           |

### Extraction Modes

| Mode          | Cost   | Description                                                                                 |
| ------------- | ------ | ------------------------------------------------------------------------------------------- |
| `static`      | Free   | Extract a value from a known JSON path in the event data.                                   |
| `ai_classify` | Free   | Classify content into predefined categories. Requires `ai_labels`.                          |
| `ai_extract`  | Free   | Extract structured fields from text. Requires `ai_labels`.                                  |
| `ai_query`    | Varies | Run a custom prompt against conversation data. Requires `model_tier` and `ai_query_prompt`. |
| `sql_expr`    | Free   | Raw SQL expression. Not available for custom metrics (security).                            |

{% hint style="warning" %}
Custom metrics cannot use `extraction_mode: "sql_expr"` due to SQL injection risk. Use `ai_query` for complex evaluations.
{% endhint %}

## Evaluate a Metric On-Demand

Test a metric definition against a specific conversation without waiting for the batch pipeline. The result is not persisted to the metric store.

```
POST /v1/{workspace_id}/metrics/{metric_key}/evaluate
```

{% tabs %}
{% tab title="curl" %}

```bash
curl -X POST \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"subject_id": "CA3db8587c7ac832dbca306287ef5f42f9"}' \
  "https://api.platform.amigo.ai/v1/$WORKSPACE_ID/metrics/empathy_score/evaluate"
```

{% endtab %}

{% tab title="Python" %}

```python
resp = client.post(
    f"/v1/{workspace_id}/metrics/empathy_score/evaluate",
    json={"subject_id": "CA3db8587c7ac832dbca306287ef5f42f9"},
)
result = resp.json()
print(f"Empathy: {result['metric']['value']}")
```

{% endtab %}

{% tab title="Forge CLI" %}

```bash
forge platform metrics evaluate empathy_score CA3db8587c7ac832dbca306287ef5f42f9 --env myorg
```

{% endtab %}
{% endtabs %}

**Request body:**

| Field        | Type   | Description                                   |
| ------------ | ------ | --------------------------------------------- |
| `subject_id` | string | CallSid, call UUID, or simulation session ID. |

**Response:**

```json
{
  "metric": {
    "metric_key": "empathy_score",
    "metric_type": "numerical",
    "value": 7.0,
    "source": "production",
    "entity_type": "call",
    "entity_id": "CA3db8587c7ac832dbca306287ef5f42f9",
    "period_start": "2026-04-28T00:00:00Z",
    "period_end": "2026-04-29T00:00:00Z",
    "event_count": 1,
    "unit": "score",
    "computed_at": "2026-04-28T21:11:42.847442Z"
  },
  "persisted": false,
  "evaluated_at": "2026-04-28T21:11:42.847442Z"
}
```

{% hint style="info" %}
On-demand evaluation currently supports `ai_query` metrics on `call_intelligence` source only. The call must have a conversation summary available (calls that ended before the summary was generated return 404).
{% endhint %}

**Error responses:**

| Status | Meaning                                                                                     |
| ------ | ------------------------------------------------------------------------------------------- |
| 404    | Metric definition not found, or call intelligence snapshot not found for the given subject. |
| 422    | Metric cannot be evaluated on demand (unsupported extraction mode or source).               |
| 502    | The analytics query failed. Retry after a few seconds.                                      |
| 503    | Analytics warehouse not configured for this workspace.                                      |

## Query Metric Values

Computed metric values are available through four query endpoints. Values are computed by the batch pipeline (hourly for AI metrics, continuously for static metrics) and synced to the operational database.

### List Latest Values

```
GET /v1/{workspace_id}/metrics
```

Returns the most recent value for each metric.

| Parameter     | Type   | Default      | Description                                                  |
| ------------- | ------ | ------------ | ------------------------------------------------------------ |
| `source`      | string | `production` | `production`, `simulation`, or `all`                         |
| `scope`       | string | `aggregate`  | `aggregate` (workspace-level), `entity` (per-call), or `all` |
| `entity_type` | string | -            | Filter by entity type (e.g., `call`)                         |
| `entity_id`   | string | -            | Filter by specific entity                                    |
| `service_id`  | string | -            | Filter by service                                            |
| `session_id`  | string | -            | Filter by session (CallSid)                                  |

### Get Metric Values

```
GET /v1/{workspace_id}/metrics/{metric_key}
```

Returns values for a specific metric with optional time range.

| Parameter   | Type     | Default | Description                  |
| ----------- | -------- | ------- | ---------------------------- |
| `date_from` | datetime | -       | Start of range (ISO-8601)    |
| `date_to`   | datetime | -       | End of range (ISO-8601)      |
| `limit`     | integer  | 90      | Max values to return (1-365) |

### Get Metric Trend

```
GET /v1/{workspace_id}/metrics/{metric_key}/trend
```

Returns time-series data for dashboard visualizations.

| Parameter | Type    | Default | Description             |
| --------- | ------- | ------- | ----------------------- |
| `days`    | integer | 30      | Lookback window (1-365) |

### Metric Catalog

```
GET /v1/{workspace_id}/metrics/catalog
```

Lists all available metrics (built-in and custom) with their configuration.

## Custom Metric Examples

### Conversation Empathy Score

A numerical metric that rates how empathetic the agent was during each conversation.

```json
{
  "key": "empathy_score",
  "name": "Agent Empathy",
  "metric_type": "numerical",
  "source": "call_intelligence",
  "event_types": ["call_intelligence.row"],
  "extraction_mode": "ai_query",
  "model_tier": "balanced",
  "ai_query_prompt": "Rate how empathetic the agent was during this conversation on a scale of 1-10. Consider: did the agent acknowledge patient concerns, show understanding, and respond with appropriate emotional sensitivity? Output ONLY a single integer from 1 to 10.",
  "aggregation": "avg",
  "unit": "score",
  "valid_range_min": 1.0,
  "valid_range_max": 10.0
}
```

### Protocol Adherence Check

A boolean metric that checks whether the agent followed the required clinical protocol.

```json
{
  "key": "protocol_followed",
  "name": "Protocol Adherence",
  "metric_type": "boolean",
  "source": "call_intelligence",
  "event_types": ["call_intelligence.row"],
  "extraction_mode": "ai_query",
  "model_tier": "balanced",
  "ai_query_prompt": "Did the agent follow the standard clinical intake protocol? The protocol requires: (1) verify patient identity, (2) confirm reason for call, (3) collect or verify insurance information, (4) schedule or confirm next steps. Respond with exactly: true or false.",
  "aggregation": "rate",
  "unit": "rate"
}
```

### Call Topic Classification

A categorical metric that classifies the primary reason for each call.

```json
{
  "key": "call_topic",
  "name": "Call Topic",
  "metric_type": "categorical",
  "source": "call_intelligence",
  "event_types": ["call_intelligence.row"],
  "extraction_mode": "ai_query",
  "model_tier": "fast",
  "ai_query_prompt": "What was the primary topic of this conversation? Respond with exactly one of: scheduling, billing, clinical_question, prescription, referral, complaint, other.",
  "aggregation": "count",
  "unit": "count",
  "categories": ["scheduling", "billing", "clinical_question", "prescription", "referral", "complaint", "other"]
}
```

## Lifecycle

Custom metrics follow this lifecycle:

1. **Define** - Create the metric definition via `PUT /settings/metrics`.
2. **Test** - Use `POST /metrics/{key}/evaluate` to test against individual conversations. Iterate on the prompt until the scores match your expectations.
3. **Deploy** - Once the definition is saved, the batch pipeline picks it up automatically on the next hourly run. No deployment or code change needed.
4. **Query** - After the first pipeline run, values appear in `GET /metrics/{key}` and the metric catalog. Dashboard panels can visualize the new metric immediately.
5. **Iterate** - Update the prompt or extraction settings via `PUT /settings/metrics`. Changes take effect on the next pipeline run.

{% hint style="info" %}
The batch pipeline runs hourly. After defining a new metric, the first computed values appear within one hour. Use the on-demand evaluate endpoint for immediate testing.
{% endhint %}

## Limits

| Limit                                      | Value             |
| ------------------------------------------ | ----------------- |
| Custom metrics per workspace               | 50                |
| `ai_query_prompt` length                   | 8,000 characters  |
| Conversations evaluated per metric per run | 500 (most recent) |
| Evaluation lookback window                 | 24 hours          |

## Forge CLI

The [Forge CLI](https://github.com/amigo-ai/agent-forge) provides three commands for working with metrics:

```bash
# View all metric definitions (built-in + custom)
forge platform metrics settings --env myorg

# Create or update custom metrics
forge platform metrics define --env myorg --body '{"definitions": [...]}'

# Evaluate a metric against a specific call
forge platform metrics evaluate empathy_score CA3db8587... --env myorg

# List latest metric values
forge platform metrics list --env myorg

# Get values for a specific metric
forge platform metrics get empathy_score --env myorg

# View time-series trend
forge platform metrics trend empathy_score --days 14 --env myorg
```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.amigo.ai/developer-guide/platform-api/platform-api/metric-store.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.