> For the complete documentation index, see [llms.txt](https://docs.amigo.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.amigo.ai/developer-guide/classic-api/core-api/metrics.md). # Metrics Manage organization-scoped measurement criteria for evaluating conversation quality. This page documents the core API endpoints for creating, configuring, evaluating, and retrieving metrics. {% hint style="info" %} **What are Metrics?** Metrics define measurable criteria for assessing conversation quality. They work with the simulation system for automated testing: when a simulation run completes, configured metrics are automatically evaluated against the conversation. Metrics can also be evaluated manually against any completed conversation through the Evaluate endpoint. {% endhint %} ## Overview Metrics are evaluation criteria that produce typed results (boolean, numerical, or categorical) when applied to a conversation. They serve three purposes: * **Measure conversation quality** after sessions complete (post-session evaluation). * **Automate testing** through the simulation system, where metrics score simulated conversations. * **Manual evaluation** of specific conversations on demand. ### Metric Value Types Each metric defines a value type that determines the shape of its evaluation results: | Type | Description | Configuration | | ------------- | -------------------------------- | ---------------------------------------- | | `boolean` | True/false evaluation | No additional config needed | | `numerical` | Numeric score within a range | Requires `lower_bound` and `upper_bound` | | `categorical` | One of a set of named categories | Requires a `categories` list | **Examples:** ```json // Boolean: "Did the agent follow the safety protocol?" { "type": "boolean" } // Numerical: "Rate empathy on a 1-10 scale" { "type": "numerical", "lower_bound": 1, "upper_bound": 10 } // Categorical: "Classify the conversation outcome" { "type": "categorical", "categories": ["resolved", "escalated", "abandoned"] } ``` ### Evaluation Sources Metric evaluation results include a `source_type` that indicates how the evaluation was triggered: | Source Type | Description | | -------------- | ------------------------------------------------- | | `post-session` | Automatically evaluated after a conversation ends | | `simulation` | Evaluated as part of a simulation test run | | `manual` | Triggered via the Evaluate Metrics API endpoint | ### Metric Versioning Metrics support version tracking. When you change a metric's evaluation definition (description and metric value), you create a new version rather than overwriting the existing one. This lets teams evolve scoring criteria over time while keeping a history of how metrics were defined at each point. Version history is read through the [Get Metric Versions endpoint](#get-metric-versions) (`GET /v1/{organization}/metric/{metric_id}/version`); pass `version=latest` to retrieve the current version. Evaluation results are tagged with the metric version they were scored against, so you can compare results across versions accurately. When creating a metric, the initial evaluation definition (description and metric value) is supplied in `initial_version_metric_value` and stored as version 1. Later changes to the definition are made by [creating a new metric version](#create-a-metric-version). ### Typical Lifecycle 1. Create a metric with a name, description, value type, and service associations. 2. Apply the metric to services so it is evaluated for their conversations. 3. Optionally use the metric in simulation unit tests for automated quality checks. 4. Evaluate manually against specific conversations as needed. 5. Retrieve evaluation results to analyze conversation quality trends. 6. Create new metric versions as the evaluation criteria evolve. Each version is added to the metric's history. {% hint style="warning" %} A metric's name cannot be changed after creation. To change the name, create a new metric and delete the old one. {% endhint %} ### Rate Limits | Endpoint | Limit | | ----------------------- | ----------------------- | | Create a metric | 100 requests per minute | | List metrics | 20 requests per minute | | Search metrics | 20 requests per minute | | Update a metric | 20 requests per minute | | Delete a metric | 100 requests per minute | | Evaluate metrics | 50 requests per minute | | Get evaluation results | 10 requests per minute | | Create a metric version | 60 requests per minute | | Get metric versions | 60 requests per minute | ## Create a Metric Create a new metric with a name, service associations, and an initial version that defines the description and metric value. The metric name must be unique within the organization. The initial evaluation definition is supplied in the required `initial_version_metric_value` object, which contains `description` and `metric_value`. {% tabs %} {% tab title="cURL" %} ```bash curl --request POST \ --url 'https://api.amigo.ai/v1//metric/' \ --header 'Authorization: Bearer ' \ --header 'Accept: application/json' \ --header 'Content-Type: application/json' \ --data '{ "name": "Empathy Score", "applied_to_services": ["6618791275130b73714e8d1c"], "additional_notes": "Score of 7+ indicates good empathy. Below 5 needs review.", "tags": {"category": "quality", "team": "support"}, "initial_version_metric_value": { "description": "Measures how empathetic the agent was during the conversation", "metric_value": { "type": "numerical", "lower_bound": 1, "upper_bound": 10 } } }' ``` {% endtab %} {% tab title="Python SDK" %} ```python from amigo_sdk import AmigoClient with AmigoClient() as client: result = client.metric.create( name="Empathy Score", applied_to_services=["6618791275130b73714e8d1c"], additional_notes="Score of 7+ indicates good empathy. Below 5 needs review.", tags={"category": "quality", "team": "support"}, initial_version_metric_value={ "description": "Measures how empathetic the agent was during the conversation", "metric_value": { "type": "numerical", "lower_bound": 1, "upper_bound": 10, }, }, ) print(f"Created metric: {result.id}") ``` {% endtab %} {% tab title="TypeScript SDK" %} ```typescript import { AmigoClient } from "@amigo-ai/sdk"; const client = new AmigoClient({ ...config }); const result = await client.metric.create({ name: "Empathy Score", applied_to_services: ["6618791275130b73714e8d1c"], additional_notes: "Score of 7+ indicates good empathy. Below 5 needs review.", tags: { category: "quality", team: "support" }, initial_version_metric_value: { description: "Measures how empathetic the agent was during the conversation", metric_value: { type: "numerical", lower_bound: 1, upper_bound: 10, }, }, }); console.log(`Created metric: ${result.id}`); ``` {% endtab %} {% endtabs %} **Response (201):** ```json { "id": "6618791275130b73714e8d1c" } ``` {% openapi src="" path="/v1/{organization}/metric/" method="post" %} {% endopenapi %} ## List Metrics Retrieve metrics matching the given filters. Only metrics the authenticated user has `Metric:GetMetric` permission for are returned. Common filters: * `id=` (repeatable): filter by specific IDs * `applied_to_service=` (repeatable): filter by associated service * `type=boolean|numerical|categorical` (repeatable): filter by metric value type * `is_deleted=true|false` (default `false`): include deleted metrics * `creator=` (repeatable): filter by creator * `tag=key:value` (repeatable; `value` may be `*` for any value, or empty for null) * `sort_by=+updated_at|-updated_at` (repeatable) * `limit` (0-50, default 50), `continuation_token` (int, default 0) {% tabs %} {% tab title="cURL" %} ```bash curl --request GET \ --url 'https://api.amigo.ai/v1//metric/?limit=10&type=numerical' \ --header 'Authorization: Bearer ' \ --header 'Accept: application/json' ``` {% endtab %} {% tab title="Python SDK" %} ```python from amigo_sdk import AmigoClient with AmigoClient() as client: result = client.metric.list(limit=10, type=["numerical"]) for metric in result.metrics: print(f"{metric.name} (ID: {metric.id}, type: {metric.metric_value_type})") ``` {% endtab %} {% tab title="TypeScript SDK" %} ```typescript import { AmigoClient } from "@amigo-ai/sdk"; const client = new AmigoClient({ ...config }); const result = await client.metric.list({ limit: 10, type: ["numerical"], }); result.metrics?.forEach((metric) => { console.log(`${metric.name} (ID: ${metric.id})`); }); ``` {% endtab %} {% endtabs %} {% openapi src="" path="/v1/{organization}/metric/" method="get" %} {% endopenapi %} ## Search Metrics Search for metrics by text query. The query matches against metric names. Returns the top 50 results sorted by relevance. {% openapi src="" path="/v1/{organization}/metric/search/" method="get" %} {% endopenapi %} ## Update a Metric Update a metric's service associations, notes, or tags. All fields are optional. Only the fields you provide are updated. {% hint style="warning" %} A metric's **name** cannot be changed after creation. Its evaluation definition (description and metric value) is changed by [creating a new metric version](#create-a-metric-version), not through this endpoint. {% endhint %} ```bash curl --request POST \ --url 'https://api.amigo.ai/v1//metric//' \ --header 'Authorization: Bearer ' \ --header 'Accept: application/json' \ --header 'Content-Type: application/json' \ --data '{ "additional_notes": "Score of 7+ indicates good empathy. Below 5 needs review.", "tags": {"category": "quality", "status": "reviewed"} }' ``` {% openapi src="" path="/v1/{organization}/metric/{metric\_id}/" method="post" %} {% endopenapi %} ## Create a Metric Version Create a new version of a metric's evaluation definition. The request body contains the two fields that make up the definition: `description` (what the metric is and how the result should be generated) and `metric_value` (the value type configuration). The metric value **type** cannot be changed between versions; attempting to do so returns HTTP 400. The response returns the numerical version just created; for the first version of a metric this is `1`. **Permission required:** `Metric:ModifyMetric` ```bash curl --request POST \ --url 'https://api.amigo.ai/v1//metric//version/' \ --header 'Authorization: Bearer ' \ --header 'Accept: application/json' \ --header 'Content-Type: application/json' \ --data '{ "description": "Measures how empathetic the agent was during the conversation, weighting acknowledgment of user concerns", "metric_value": { "type": "numerical", "lower_bound": 1, "upper_bound": 10 } }' ``` {% openapi src="" path="/v1/{organization}/metric/{metric\_id}/version/" method="post" %} {% endopenapi %} ## Get Metric Versions Retrieve the versions of a metric. The `version` query parameter accepts an exact version number, `latest` for the current version, or an inclusive range separated by `-` (for example, `2-5` or `1-latest`). Results can be paged with `limit` (max 10) and `continuation_token`, and sorted by `version`. **Permission required:** `Metric:GetMetric` ```bash curl --request GET \ --url 'https://api.amigo.ai/v1//metric//version?version=latest' \ --header 'Authorization: Bearer ' \ --header 'Accept: application/json' ``` {% openapi src="" path="/v1/{organization}/metric/{metric\_id}/version" method="get" %} {% endopenapi %} ## Delete a Metric Delete a metric. The metric is no longer modifiable or manually evaluable, but remains stored, and existing services or simulations that evaluate this metric continue to do so. You can still retrieve it by ID to view its specifics. {% hint style="warning" %} Deletion will fail with HTTP 400 if the metric is currently used in any simulation unit tests. Remove the metric from all simulation unit tests before deleting. {% endhint %} {% openapi src="" path="/v1/{organization}/metric/{metric\_id}/" method="delete" %} {% endopenapi %} ## Evaluate Metrics Evaluate one or more metrics against a completed conversation. Results are stored and retrievable through the Get Evaluation Results endpoint with a source type of `manual`. If the same conversation has been manually evaluated for the same metric previously, the previous evaluation result is overwritten. {% tabs %} {% tab title="cURL" %} ```bash curl --request POST \ --url 'https://api.amigo.ai/v1//metric/evaluate' \ --header 'Authorization: Bearer ' \ --header 'Accept: application/json' \ --header 'Content-Type: application/json' \ --data '{ "metric_ids": ["6618791275130b73714e8d1c", "6618791275130b73714e8d1d"], "conversation_id": "6618791275130b73714eaaaa" }' ``` {% endtab %} {% tab title="Python SDK" %} ```python from amigo_sdk import AmigoClient with AmigoClient() as client: result = client.metric.evaluate( metric_ids=["6618791275130b73714e8d1c"], conversation_id="6618791275130b73714eaaaa", ) for evaluation in result.metrics: print(f"Metric: {evaluation.metric_id}, Result: {evaluation.result}") ``` {% endtab %} {% tab title="TypeScript SDK" %} ```typescript import { AmigoClient } from "@amigo-ai/sdk"; const client = new AmigoClient({ ...config }); const result = await client.metric.evaluate({ metric_ids: ["6618791275130b73714e8d1c"], conversation_id: "6618791275130b73714eaaaa", }); result.metrics?.forEach((evaluation) => { console.log(`Metric: ${evaluation.metric_id}, Result: ${evaluation.result}`); }); ``` {% endtab %} {% endtabs %} Request body: | Field | Type | Required | Description | | ---------------------------- | -------------- | -------- | -------------------------------------------------------------------------------- | | `metric_ids` | array\ | Yes | IDs of metrics to evaluate (1-10, must be unique) | | `conversation_id` | string | Yes | ID of the completed conversation to evaluate | | `evaluate_to_interaction_id` | string | No | If specified, only messages up to (and including) this interaction are evaluated | {% hint style="info" %} The conversation must have at least one interaction, or the request returns HTTP 400. {% endhint %} {% openapi src="" path="/v1/{organization}/metric/evaluate" method="post" %} {% endopenapi %} ## Get Evaluation Results Retrieve metric evaluation results matching the given filters. Results include the metric ID, evaluated value, source type, and associated conversation or simulation run. Common filters: * `metric_id=` (repeatable, deprecated): filter by metric * `metric_id_and_version=-` (repeatable): filter by metric and version; leave `` empty to match any version * `source_type=post-session|manual|simulation` (repeatable): filter by evaluation source * `conversation_id=` (repeatable): filter by conversation (for `post-session` and `manual` sources) * `simulation_unit_test_set_run_id=` (repeatable): filter by simulation run (for `simulation` source) * `timestamp_after=`: only results evaluated on or after this time * `timestamp_before=`: only results evaluated on or before this time * `sort_by=+created_at|-created_at` (repeatable) * `limit` (0-500, default 500), `continuation_token` (int, default 0) {% tabs %} {% tab title="cURL" %} ```bash curl --request GET \ --url 'https://api.amigo.ai/v1//metric/metric_evaluation_result?metric_id=6618791275130b73714e8d1c&source_type=manual&limit=20' \ --header 'Authorization: Bearer ' \ --header 'Accept: application/json' ``` {% endtab %} {% tab title="Python SDK" %} ```python from amigo_sdk import AmigoClient with AmigoClient() as client: result = client.metric.get_evaluation_results( metric_id=["6618791275130b73714e8d1c"], source_type=["manual"], limit=20, ) for r in result.metric_evaluation_results: print(f"Metric: {r.metric_id}, Result: {r.result}, Source: {r.source_type}") ``` {% endtab %} {% tab title="TypeScript SDK" %} ```typescript import { AmigoClient } from "@amigo-ai/sdk"; const client = new AmigoClient({ ...config }); const result = await client.metric.getEvaluationResults({ metric_id: ["6618791275130b73714e8d1c"], source_type: ["manual"], limit: 20, }); result.metric_evaluation_results?.forEach((r) => { console.log(`Metric: ${r.metric_id}, Result: ${r.result}`); }); ``` {% endtab %} {% endtabs %} {% openapi src="" path="/v1/{organization}/metric/metric\_evaluation\_result" method="get" %} {% endopenapi %} ## Related * Data Access → [Metric Tables](/developer-guide/classic-api/data-access/organization-tables/metric.md) * Data Access → [Simulation Tables](/developer-guide/classic-api/data-access/organization-tables/simulation.md) * Classic API -> [Services](/developer-guide/classic-api/core-api/services.md) (metric application through services) * Getting Started → [Authentication](/developer-guide/getting-started/authentication.md) --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.amigo.ai/developer-guide/classic-api/core-api/metrics.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.