# Metrics Manage organization-scoped measurement criteria for evaluating conversation quality. This page documents the core API endpoints for creating, configuring, evaluating, and retrieving metrics. {% hint style="info" %} **What are Metrics?** Metrics define measurable criteria for assessing conversation quality. They work with the simulation system for automated testing - when a simulation run completes, configured metrics are automatically evaluated against the conversation. Metrics can also be evaluated manually against any completed conversation via the Evaluate endpoint. {% endhint %} ## Overview Metrics are evaluation criteria that produce typed results (boolean, numerical, or categorical) when applied to a conversation. They are used to: * **Measure conversation quality** after sessions complete (post-session evaluation). * **Automate testing** via the simulation system, where metrics score simulated conversations. * **Manual evaluation** of specific conversations on demand. ### Metric Value Types Each metric defines a value type that determines the shape of its evaluation results: | Type | Description | Configuration | | ------------- | -------------------------------- | ---------------------------------------- | | `boolean` | True/false evaluation | No additional config needed | | `numerical` | Numeric score within a range | Requires `lower_bound` and `upper_bound` | | `categorical` | One of a set of named categories | Requires a `categories` list | **Examples:** ```json // Boolean: "Did the agent follow the safety protocol?" { "type": "boolean" } // Numerical: "Rate empathy on a 1-10 scale" { "type": "numerical", "lower_bound": 1, "upper_bound": 10 } // Categorical: "Classify the conversation outcome" { "type": "categorical", "categories": ["resolved", "escalated", "abandoned"] } ``` ### Evaluation Sources Metric evaluation results include a `source_type` that indicates how the evaluation was triggered: | Source Type | Description | | -------------- | ------------------------------------------------- | | `post-session` | Automatically evaluated after a conversation ends | | `simulation` | Evaluated as part of a simulation test run | | `manual` | Triggered via the Evaluate Metrics API endpoint | ### Metric Versioning Metrics support version tracking. When you update a metric's evaluation definition (description, value type configuration), the system creates a new version rather than overwriting the existing one. This lets teams evolve scoring criteria over time while maintaining a history of how metrics were defined at each point. Each metric has a `latest_version` field that tracks the current version number. Evaluation results are tagged with the metric version they were scored against, so you can compare results across versions accurately. When creating a metric, the initial evaluation definition (description and metric value) is stored as version 1. Subsequent updates to these fields create new versions automatically. ### Typical Lifecycle 1. Create a metric with a name, description, value type, and service associations. 2. Apply the metric to services so it is evaluated for their conversations. 3. Optionally use the metric in simulation unit tests for automated quality checks. 4. Evaluate manually against specific conversations as needed. 5. Retrieve evaluation results to analyze conversation quality trends. 6. Update the metric's evaluation criteria as needed - each change creates a new version while preserving history. {% hint style="warning" %} A metric's name cannot be changed after creation. To change the name, create a new metric and delete the old one. {% endhint %} ## Create a Metric Create a new metric with a name, description, value type, and service associations. The metric name must be unique within the organization. {% tabs %} {% tab title="cURL" %} ```bash curl --request POST \ --url 'https://api.amigo.ai/v1//metric/' \ --header 'Authorization: Bearer ' \ --header 'Accept: application/json' \ --header 'Content-Type: application/json' \ --data '{ "name": "Empathy Score", "description": "Measures how empathetic the agent was during the conversation", "applied_to_services": ["6618791275130b73714e8d1c"], "additional_notes": "Score of 7+ indicates good empathy. Below 5 needs review.", "tags": {"category": "quality", "team": "support"}, "metric_value": { "type": "numerical", "lower_bound": 1, "upper_bound": 10 } }' ``` {% endtab %} {% tab title="Python SDK" %} ```python from amigo_sdk import AmigoClient with AmigoClient() as client: result = client.metric.create( name="Empathy Score", description="Measures how empathetic the agent was during the conversation", applied_to_services=["6618791275130b73714e8d1c"], additional_notes="Score of 7+ indicates good empathy. Below 5 needs review.", tags={"category": "quality", "team": "support"}, metric_value={ "type": "numerical", "lower_bound": 1, "upper_bound": 10, }, ) print(f"Created metric: {result.id}") ``` {% endtab %} {% tab title="TypeScript SDK" %} ```typescript import { AmigoClient } from "@amigo-ai/sdk"; const client = new AmigoClient({ ...config }); const result = await client.metric.create({ name: "Empathy Score", description: "Measures how empathetic the agent was during the conversation", applied_to_services: ["6618791275130b73714e8d1c"], additional_notes: "Score of 7+ indicates good empathy. Below 5 needs review.", tags: { category: "quality", team: "support" }, metric_value: { type: "numerical", lower_bound: 1, upper_bound: 10, }, }); console.log(`Created metric: ${result.id}`); ``` {% endtab %} {% endtabs %} **Response (201):** ```json { "id": "6618791275130b73714e8d1c" } ``` {% hint style="info" %} **Rate Limit:** 100 requests per minute. {% endhint %} {% openapi src="" path="/v1/{organization}/metric/" method="post" %} {% endopenapi %} ## List Metrics Retrieve metrics matching the given filters. Only metrics the authenticated user has `Metric:GetMetric` permission for are returned. Common filters: * `id=` (repeatable) - filter by specific IDs * `applied_to_service=` (repeatable) - filter by associated service * `type=boolean|numerical|categorical` (repeatable) - filter by metric value type * `is_deleted=true|false` (default `false`) - include deleted metrics * `creator=` (repeatable) - filter by creator * `tag=key:value` (repeatable; `value` may be `*` for any value or empty for null) * `sort_by=+updated_at|-updated_at` (repeatable) * `limit` (0-50, default 50), `continuation_token` (int, default 0) {% tabs %} {% tab title="cURL" %} ```bash curl --request GET \ --url 'https://api.amigo.ai/v1//metric/?limit=10&type=numerical' \ --header 'Authorization: Bearer ' \ --header 'Accept: application/json' ``` {% endtab %} {% tab title="Python SDK" %} ```python from amigo_sdk import AmigoClient with AmigoClient() as client: result = client.metric.list(limit=10, type=["numerical"]) for metric in result.metrics: print(f"{metric.name} (ID: {metric.id}, type: {metric.metric_value.type})") ``` {% endtab %} {% tab title="TypeScript SDK" %} ```typescript import { AmigoClient } from "@amigo-ai/sdk"; const client = new AmigoClient({ ...config }); const result = await client.metric.list({ limit: 10, type: ["numerical"], }); result.metrics?.forEach((metric) => { console.log(`${metric.name} (ID: ${metric.id})`); }); ``` {% endtab %} {% endtabs %} {% hint style="info" %} **Rate Limit:** 20 requests per minute. {% endhint %} {% openapi src="" path="/v1/{organization}/metric/" method="get" %} {% endopenapi %} ## Search Metrics Search for metrics by text query. The query matches against metric names and descriptions. Returns the top 50 results sorted by relevance. {% hint style="info" %} **Rate Limit:** 20 requests per minute. {% endhint %} {% openapi src="" path="/v1/{organization}/metric/search/" method="get" %} {% endopenapi %} ## Update a Metric Update a metric's description, service associations, notes, or tags. All fields are optional; only provided fields are updated. {% hint style="warning" %} A metric's **name** and **metric value type** cannot be changed after creation. {% endhint %} ```bash curl --request POST \ --url 'https://api.amigo.ai/v1//metric//' \ --header 'Authorization: Bearer ' \ --header 'Accept: application/json' \ --header 'Content-Type: application/json' \ --data '{ "description": "Updated description of the metric", "tags": {"category": "quality", "status": "reviewed"} }' ``` {% hint style="info" %} **Rate Limit:** 20 requests per minute. {% endhint %} {% openapi src="" path="/v1/{organization}/metric/{metric\_id}/" method="post" %} {% endopenapi %} ## Delete a Metric Delete a metric. The metric becomes non-modifiable and non-usable, but remains in the database for historical reference. You can still retrieve it by ID to view metrics that were evaluated in previous conversations. {% hint style="warning" %} Deletion will fail with HTTP 400 if the metric is currently used in any simulation unit tests. Remove the metric from all simulation unit tests before deleting. {% endhint %} {% hint style="info" %} **Rate Limit:** 100 requests per minute. {% endhint %} {% openapi src="" path="/v1/{organization}/metric/{metric\_id}/" method="delete" %} {% endopenapi %} ## Evaluate Metrics Evaluate one or more metrics against a completed conversation. Results are stored and retrievable through the Get Evaluation Results endpoint with a source type of `manual`. If the same conversation has been manually evaluated for the same metric previously, the previous evaluation result is overwritten. {% tabs %} {% tab title="cURL" %} ```bash curl --request POST \ --url 'https://api.amigo.ai/v1//metric/evaluate' \ --header 'Authorization: Bearer ' \ --header 'Accept: application/json' \ --header 'Content-Type: application/json' \ --data '{ "metric_ids": ["6618791275130b73714e8d1c", "6618791275130b73714e8d1d"], "conversation_id": "6618791275130b73714eaaaa" }' ``` {% endtab %} {% tab title="Python SDK" %} ```python from amigo_sdk import AmigoClient with AmigoClient() as client: result = client.metric.evaluate( metric_ids=["6618791275130b73714e8d1c"], conversation_id="6618791275130b73714eaaaa", ) for evaluation in result.metrics: print(f"Metric: {evaluation.metric_id}, Result: {evaluation.result}") ``` {% endtab %} {% tab title="TypeScript SDK" %} ```typescript import { AmigoClient } from "@amigo-ai/sdk"; const client = new AmigoClient({ ...config }); const result = await client.metric.evaluate({ metric_ids: ["6618791275130b73714e8d1c"], conversation_id: "6618791275130b73714eaaaa", }); result.metrics?.forEach((evaluation) => { console.log(`Metric: ${evaluation.metric_id}, Result: ${evaluation.result}`); }); ``` {% endtab %} {% endtabs %} Request body: | Field | Type | Required | Description | | ---------------------------- | -------------- | -------- | -------------------------------------------------------------------------------- | | `metric_ids` | array\ | Yes | IDs of metrics to evaluate (1-10, must be unique) | | `conversation_id` | string | Yes | ID of the completed conversation to evaluate | | `evaluate_to_interaction_id` | string | No | If specified, only messages up to (and including) this interaction are evaluated | {% hint style="info" %} The conversation must have at least one interaction, or the request returns HTTP 400. {% endhint %} {% hint style="info" %} **Rate Limit:** 50 requests per minute. {% endhint %} {% openapi src="" path="/v1/{organization}/metric/evaluate" method="post" %} {% endopenapi %} ## Get Evaluation Results Retrieve metric evaluation results matching the given filters. Results include the metric ID, evaluated value, source type, and associated conversation or simulation run. Common filters: * `metric_id=` (repeatable) - filter by metric * `source_type=post-session|manual|simulation` (repeatable) - filter by evaluation source * `conversation_id=` (repeatable) - filter by conversation (for `post-session` and `manual` sources) * `simulation_unit_test_set_run_id=` (repeatable) - filter by simulation run (for `simulation` source) * `timestamp_after=` - only results evaluated on or after this time * `timestamp_before=` - only results evaluated on or before this time * `sort_by=+created_at|-created_at` (repeatable) * `limit` (0-500, default 500), `continuation_token` (int, default 0) {% tabs %} {% tab title="cURL" %} ```bash curl --request GET \ --url 'https://api.amigo.ai/v1//metric/metric_evaluation_result?metric_id=6618791275130b73714e8d1c&source_type=manual&limit=20' \ --header 'Authorization: Bearer ' \ --header 'Accept: application/json' ``` {% endtab %} {% tab title="Python SDK" %} ```python from amigo_sdk import AmigoClient with AmigoClient() as client: result = client.metric.get_evaluation_results( metric_id=["6618791275130b73714e8d1c"], source_type=["manual"], limit=20, ) for r in result.metric_evaluation_results: print(f"Metric: {r.metric_id}, Result: {r.result}, Source: {r.source_type}") ``` {% endtab %} {% tab title="TypeScript SDK" %} ```typescript import { AmigoClient } from "@amigo-ai/sdk"; const client = new AmigoClient({ ...config }); const result = await client.metric.getEvaluationResults({ metric_id: ["6618791275130b73714e8d1c"], source_type: ["manual"], limit: 20, }); result.metric_evaluation_results?.forEach((r) => { console.log(`Metric: ${r.metric_id}, Result: ${r.result}`); }); ``` {% endtab %} {% endtabs %} {% hint style="info" %} **Rate Limit:** 10 requests per minute. {% endhint %} {% openapi src="" path="/v1/{organization}/metric/metric\_evaluation\_result" method="get" %} {% endopenapi %} ## Related * Data Access -> [Metric Tables](https://docs.amigo.ai/developer-guide/classic-api/data-access/organization-tables/metric) * Data Access -> [Simulation Tables](https://docs.amigo.ai/developer-guide/classic-api/data-access/organization-tables/simulation) * Core API -> [Services](https://docs.amigo.ai/developer-guide/classic-api/core-api/services) (metric application via services) * Getting Started -> [Authentication](https://docs.amigo.ai/developer-guide/getting-started/authentication)