Metrics

Define and evaluate conversation quality metrics (boolean, numerical, or categorical) with automated scoring.

Manage organization-scoped measurement criteria for evaluating conversation quality. This page documents the core API endpoints for creating, configuring, evaluating, and retrieving metrics.

What are Metrics? Metrics define measurable criteria for assessing conversation quality. They work with the simulation system for automated testing: when a simulation run completes, configured metrics are automatically evaluated against the conversation. Metrics can also be evaluated manually against any completed conversation through the Evaluate endpoint.

Overview

Metrics are evaluation criteria that produce typed results (boolean, numerical, or categorical) when applied to a conversation. They serve three purposes:

Measure conversation quality after sessions complete (post-session evaluation).
Automate testing through the simulation system, where metrics score simulated conversations.
Manual evaluation of specific conversations on demand.

Metric Value Types

Each metric defines a value type that determines the shape of its evaluation results:

Type

Description

Configuration

boolean

True/false evaluation

No additional config needed

numerical

Numeric score within a range

Requires lower_bound and upper_bound

categorical

One of a set of named categories

Requires a categories list

Examples:

// Boolean: "Did the agent follow the safety protocol?"
{ "type": "boolean" }

// Numerical: "Rate empathy on a 1-10 scale"
{ "type": "numerical", "lower_bound": 1, "upper_bound": 10 }

// Categorical: "Classify the conversation outcome"
{ "type": "categorical", "categories": ["resolved", "escalated", "abandoned"] }

Evaluation Sources

Metric evaluation results include a source_type that indicates how the evaluation was triggered:

Source Type

Description

post-session

Automatically evaluated after a conversation ends

simulation

Evaluated as part of a simulation test run

manual

Triggered via the Evaluate Metrics API endpoint

Metric Versioning

Metrics support version tracking. When you change a metric's evaluation definition (description and metric value), you create a new version rather than overwriting the existing one. This lets teams evolve scoring criteria over time while keeping a history of how metrics were defined at each point.

Version history is read through the Get Metric Versions endpoint (GET /v1/{organization}/metric/{metric_id}/version); pass version=latest to retrieve the current version. Evaluation results are tagged with the metric version they were scored against, so you can compare results across versions accurately.

When creating a metric, the initial evaluation definition (description and metric value) is supplied in initial_version_metric_value and stored as version 1. Later changes to the definition are made by creating a new metric version.

Typical Lifecycle

Create a metric with a name, description, value type, and service associations.
Apply the metric to services so it is evaluated for their conversations.
Optionally use the metric in simulation unit tests for automated quality checks.
Evaluate manually against specific conversations as needed.
Retrieve evaluation results to analyze conversation quality trends.
Create new metric versions as the evaluation criteria evolve. Each version is added to the metric's history.

A metric's name cannot be changed after creation. To change the name, create a new metric and delete the old one.

Rate Limits

Endpoint

Limit

Create a metric

100 requests per minute

List metrics

20 requests per minute

Search metrics

20 requests per minute

Update a metric

20 requests per minute

Delete a metric

100 requests per minute

Evaluate metrics

50 requests per minute

Get evaluation results

10 requests per minute

Create a metric version

60 requests per minute

Get metric versions

60 requests per minute

Create a Metric

Create a new metric with a name, service associations, and an initial version that defines the description and metric value. The metric name must be unique within the organization. The initial evaluation definition is supplied in the required initial_version_metric_value object, which contains description and metric_value.

curl --request POST \
     --url 'https://api.amigo.ai/v1/<YOUR-ORG-ID>/metric/' \
     --header 'Authorization: Bearer <AUTH-TOKEN>' \
     --header 'Accept: application/json' \
     --header 'Content-Type: application/json' \
     --data '{
  "name": "Empathy Score",
  "applied_to_services": ["6618791275130b73714e8d1c"],
  "additional_notes": "Score of 7+ indicates good empathy. Below 5 needs review.",
  "tags": {"category": "quality", "team": "support"},
  "initial_version_metric_value": {
    "description": "Measures how empathetic the agent was during the conversation",
    "metric_value": {
      "type": "numerical",
      "lower_bound": 1,
      "upper_bound": 10
    }
  }
}'

from amigo_sdk import AmigoClient

with AmigoClient() as client:
    result = client.metric.create(
        name="Empathy Score",
        applied_to_services=["6618791275130b73714e8d1c"],
        additional_notes="Score of 7+ indicates good empathy. Below 5 needs review.",
        tags={"category": "quality", "team": "support"},
        initial_version_metric_value={
            "description": "Measures how empathetic the agent was during the conversation",
            "metric_value": {
                "type": "numerical",
                "lower_bound": 1,
                "upper_bound": 10,
            },
        },
    )
    print(f"Created metric: {result.id}")

import { AmigoClient } from "@amigo-ai/sdk";

const client = new AmigoClient({ ...config });
const result = await client.metric.create({
  name: "Empathy Score",
  applied_to_services: ["6618791275130b73714e8d1c"],
  additional_notes: "Score of 7+ indicates good empathy. Below 5 needs review.",
  tags: { category: "quality", team: "support" },
  initial_version_metric_value: {
    description: "Measures how empathetic the agent was during the conversation",
    metric_value: {
      type: "numerical",
      lower_bound: 1,
      upper_bound: 10,
    },
  },
});
console.log(`Created metric: ${result.id}`);

Response (201):

{
  "id": "6618791275130b73714e8d1c"
}

Create a metric

post

/v1/{organization}/metric/

Create a metric.

Permissions

Metric:CreateMetric for the metric to create.

Authorizations

AuthorizationstringRequired

The username should be set to {org_id}_{user_id}, and the password should be the Amigo issued JWT token that identifies the user.

AuthorizationstringRequired

Amigo issued JWT token that identifies an user. It's issued either after logging in through the frontend, or manually through the SignInWithAPIKey endpoint.

X-ORG-IDstringRequired

An optional organization identifier that indicates from which organization the token is issued. This is used in rare cases where the user to authenticate is making a request for resources in another organization.

Path parameters

organizationstringRequired

Header parameters

x-mongo-cluster-namestring · nullableOptional

The Mongo cluster name to perform this request in. This is usually not needed unless the organization does not exist yet in the Amigo organization infra config database.

Sec-WebSocket-Protocolstring[]OptionalDefault: []

Body

namestring · min: 1Required

The name of the metric. Must be unique within the organization.

applied_to_servicesstring[]Required

The services that the metric is applied to.

additional_notesstring · min: 1 · nullableRequired

Additional notes about the metric.

Responses

201

Succeeded

application/json

idstringRequired

The ID of the created metric.

401

Invalid authorization credentials.

403

Missing required permissions.

404

Specified organization is not found.

409

A metric with this name already exists

422

Invalid request path parameter or request body failed validation.

429

The user has exceeded the rate limit of 100 requests per minute for this endpoint.

503

The service is going through temporary maintenance.

post/v1/{organization}/metric/

POST /v1/{organization}/metric/ HTTP/1.1
Host: api.amigo.ai
Authorization: Bearer YOUR_SECRET_TOKEN
X-ORG-ID: YOUR_API_KEY
Content-Type: application/json
Accept: */*
Content-Length: 231

{
  "name": "text",
  "applied_to_services": [
    "text"
  ],
  "additional_notes": "text",
  "tags": {
    "ANY_ADDITIONAL_PROPERTY": "text"
  },
  "initial_version_metric_value": {
    "description": "text",
    "metric_value": {
      "type": "text",
      "lower_bound": 1,
      "upper_bound": 1
    }
  }
}

{
  "id": "text"
}

List Metrics

Retrieve metrics matching the given filters. Only metrics the authenticated user has Metric:GetMetric permission for are returned.

Common filters:

id=<id> (repeatable): filter by specific IDs
applied_to_service=<service_id> (repeatable): filter by associated service
type=boolean|numerical|categorical (repeatable): filter by metric value type
is_deleted=true|false (default false): include deleted metrics
creator=<org_id,user_id> (repeatable): filter by creator
tag=key:value (repeatable; value may be * for any value, or empty for null)
sort_by=+updated_at|-updated_at (repeatable)
limit (0-50, default 50), continuation_token (int, default 0)

curl --request GET \
     --url 'https://api.amigo.ai/v1/<YOUR-ORG-ID>/metric/?limit=10&type=numerical' \
     --header 'Authorization: Bearer <AUTH-TOKEN>' \
     --header 'Accept: application/json'

from amigo_sdk import AmigoClient

with AmigoClient() as client:
    result = client.metric.list(limit=10, type=["numerical"])
    for metric in result.metrics:
        print(f"{metric.name} (ID: {metric.id}, type: {metric.metric_value_type})")

import { AmigoClient } from "@amigo-ai/sdk";

const client = new AmigoClient({ ...config });
const result = await client.metric.list({
  limit: 10,
  type: ["numerical"],
});
result.metrics?.forEach((metric) => {
  console.log(`${metric.name} (ID: ${metric.id})`);
});

Get metrics

get

/v1/{organization}/metric/

Retrieve metrics that match the given filters.

Permissions

This endpoint may be impacted by the following permissions:

Only metrics that the authenticated user has Metric:GetMetric permission for will be retrieved.

Authorizations

AuthorizationstringRequired

The username should be set to {org_id}_{user_id}, and the password should be the Amigo issued JWT token that identifies the user.

AuthorizationstringRequired

Amigo issued JWT token that identifies an user. It's issued either after logging in through the frontend, or manually through the SignInWithAPIKey endpoint.

X-ORG-IDstringRequired

Path parameters

organizationstringRequired

Query parameters

idstring[]Optional

The IDs of the metrics to retrieve.

Default: []

limitinteger · max: 50Optional

The maximum number of metrics to return.

Default: 50

continuation_tokenintegerOptional

The continuation token from the previous request used to retrieve the next page of metrics.

Default: 0

applied_to_servicestring[]Optional

The IDs of the services that the metric is applied to.

Default: []

is_deletedbooleanOptional

Whether the metric is deleted.

Default: false

creatorstring[]Optional

The creators of the dynamic behavior sets.

Default: []

tagstring[]Optional

The tags of the dynamic behavior sets. Must be specified using the syntax key:value, which means to match all sets with the given key and value pair among its tags. If value is *, it means the value does not matter. If value is empty, it matches against when the value is None.

Default: []

sort_bystring[]Optional

The fields to sort the sets by. Supported fields are updated_at. Specify a + before the field name to indicate ascending sorting and - for descending sorting. Multiple fields can be specified to break ties.

Default: []

Header parameters

x-mongo-cluster-namestring · nullableOptional

The Mongo cluster name to perform this request in. This is usually not needed unless the organization does not exist yet in the Amigo organization infra config database.

Sec-WebSocket-Protocolstring[]OptionalDefault: []

Responses

200

Succeeded

application/json

has_morebooleanRequired

Whether there are more metrics to retrieve.

continuation_tokenany · nullableRequired

The continuation token to use to retrieve the next set of metrics.

401

Invalid authorization credentials.

403

Missing required permissions.

404

Specified organization is not found.

422

Invalid request path parameter or request query parameter failed validation.

429

The user has exceeded the rate limit of 20 requests per minute for this endpoint.

503

The service is going through temporary maintenance.

get/v1/{organization}/metric/

GET /v1/{organization}/metric/ HTTP/1.1
Host: api.amigo.ai
Authorization: Bearer YOUR_SECRET_TOKEN
X-ORG-ID: YOUR_API_KEY
Accept: */*

{
  "metrics": [
    {
      "id": "text",
      "org_id": "text",
      "name": "text",
      "applied_to_services": [
        "text"
      ],
      "additional_notes": "text",
      "tags": [
        {
          "key": "text",
          "value": "text"
        }
      ],
      "creator": {
        "org_id": "text",
        "user_id": "text"
      },
      "updated_by": {
        "org_id": "text",
        "user_id": "text"
      },
      "metric_value_type": "boolean",
      "is_deleted": true,
      "created_at": "2026-01-01T00:00:00.000Z",
      "updated_at": "2026-01-01T00:00:00.000Z"
    }
  ],
  "has_more": true,
  "continuation_token": null,
  "filter_values": {
    "applied_to_services_ids": [
      "text"
    ],
    "creators": [
      {
        "org_id": "text",
        "user_id": "text"
      }
    ],
    "tags": [
      "text"
    ],
    "types": [
      "text"
    ]
  }
}

Search Metrics

Search for metrics by text query. The query matches against metric names. Returns the top 50 results sorted by relevance.

Search metrics

get

/v1/{organization}/metric/search/

Search for metrics that match the given filters and contain the given query in its name. Only the top 50 results will be returned. The results will be sorted by the relevance of the search query.

Permissions

This endpoint may be impacted by the following permissions:

Only metrics that the authenticated user has Metric:GetMetric permission for will be retrieved.

Authorizations

AuthorizationstringRequired

The username should be set to {org_id}_{user_id}, and the password should be the Amigo issued JWT token that identifies the user.

AuthorizationstringRequired

Amigo issued JWT token that identifies an user. It's issued either after logging in through the frontend, or manually through the SignInWithAPIKey endpoint.

X-ORG-IDstringRequired

Path parameters

organizationstringRequired

Query parameters

querystring · min: 1Required

The query to search for. Any metrics containing the terms in its name would be returned.

applied_to_servicestring[]Optional

The IDs of the services that the metric is applied to.

Default: []

creatorstring[]Optional

The creators of the metrics. Each value must be of the format org_id,user_id.

Default: []

tagstring[]Optional

The tags of the metrics. Must be specified using the syntax key:value, which means to match all metrics with the given key and value pair among its tags. If value is *, it means the value does not matter. If value is empty, it matches against when the value is None.

Default: []

Header parameters

x-mongo-cluster-namestring · nullableOptional

The Mongo cluster name to perform this request in. This is usually not needed unless the organization does not exist yet in the Amigo organization infra config database.

Sec-WebSocket-Protocolstring[]OptionalDefault: []

Responses

200

Succeeded.

application/json

401

Invalid authorization credentials.

403

Missing required permissions.

404

Specified organization is not found.

422

Invalid request path parameter or request query parameter failed validation.

429

The user has exceeded the rate limit of 20 requests per minute for this endpoint.

503

The service is going through temporary maintenance.

get/v1/{organization}/metric/search/

GET /v1/{organization}/metric/search/?query=text HTTP/1.1
Host: api.amigo.ai
Authorization: Bearer YOUR_SECRET_TOKEN
X-ORG-ID: YOUR_API_KEY
Accept: */*

{
  "metrics": [
    {
      "_id": "text",
      "org_id": "text",
      "created_at": "2026-01-01T00:00:00.000Z",
      "updated_at": "2026-01-01T00:00:00.000Z",
      "name": "text",
      "applied_to_services": [
        "text"
      ],
      "additional_notes": "text",
      "tags": [
        {
          "key": "text",
          "value": "text"
        }
      ],
      "creator": {
        "org_id": "text",
        "user_id": "text"
      },
      "updated_by": {
        "org_id": "text",
        "user_id": "text"
      },
      "metric_value_type": "boolean",
      "is_deleted": true
    }
  ]
}

Update a Metric

Update a metric's service associations, notes, or tags. All fields are optional. Only the fields you provide are updated.

A metric's name cannot be changed after creation. Its evaluation definition (description and metric value) is changed by creating a new metric version, not through this endpoint.

curl --request POST \
     --url 'https://api.amigo.ai/v1/<YOUR-ORG-ID>/metric/<METRIC-ID>/' \
     --header 'Authorization: Bearer <AUTH-TOKEN>' \
     --header 'Accept: application/json' \
     --header 'Content-Type: application/json' \
     --data '{
  "additional_notes": "Score of 7+ indicates good empathy. Below 5 needs review.",
  "tags": {"category": "quality", "status": "reviewed"}
}'

Update a metric

post

/v1/{organization}/metric/{metric_id}/

Update properties of a metric. The metric's name and metric values cannot be updated.

Permissions

This endpoint requires the following permissions:

Metric:ModifyMetric for the metric to modify.

Authorizations

AuthorizationstringRequired

The username should be set to {org_id}_{user_id}, and the password should be the Amigo issued JWT token that identifies the user.

AuthorizationstringRequired

Amigo issued JWT token that identifies an user. It's issued either after logging in through the frontend, or manually through the SignInWithAPIKey endpoint.

X-ORG-IDstringRequired

Path parameters

organizationstringRequired

metric_idstringRequired

The ID of the metric to update.

Pattern: ^[a-f0-9]{24}$

Header parameters

x-mongo-cluster-namestring · nullableOptional

The Mongo cluster name to perform this request in. This is usually not needed unless the organization does not exist yet in the Amigo organization infra config database.

Sec-WebSocket-Protocolstring[]OptionalDefault: []

Body

applied_to_servicesstring[] · nullableOptional

The services that the metric is applied to. Only updated if set.

additional_notesany of · nullableOptional

Additional notes about the metric. Only updated if set.

Default: {}

string · min: 1Optional

object · _NotSetOptional

A specific type to indicate that a field is not set in the request.

Responses

200

Succeeded

application/json

anyOptional

401

Invalid authorization credentials.

403

Missing required permissions.

404

Specified organization or metric is not found.

422

Invalid request path parameter or request body failed validation.

429

The user has exceeded the rate limit of 20 requests per minute for this endpoint.

503

The service is going through temporary maintenance.

post/v1/{organization}/metric/{metric_id}/

POST /v1/{organization}/metric/{metric_id}/ HTTP/1.1
Host: api.amigo.ai
Authorization: Bearer YOUR_SECRET_TOKEN
X-ORG-ID: YOUR_API_KEY
Content-Type: application/json
Accept: */*
Content-Length: 100

{
  "applied_to_services": [
    "text"
  ],
  "additional_notes": "text",
  "tags": {
    "ANY_ADDITIONAL_PROPERTY": "text"
  }
}

No content

Create a Metric Version

Create a new version of a metric's evaluation definition. The request body contains the two fields that make up the definition: description (what the metric is and how the result should be generated) and metric_value (the value type configuration). The metric value type cannot be changed between versions; attempting to do so returns HTTP 400. The response returns the numerical version just created; for the first version of a metric this is 1.

Permission required: Metric:ModifyMetric

curl --request POST \
     --url 'https://api.amigo.ai/v1/<YOUR-ORG-ID>/metric/<METRIC-ID>/version/' \
     --header 'Authorization: Bearer <AUTH-TOKEN>' \
     --header 'Accept: application/json' \
     --header 'Content-Type: application/json' \
     --data '{
  "description": "Measures how empathetic the agent was during the conversation, weighting acknowledgment of user concerns",
  "metric_value": {
    "type": "numerical",
    "lower_bound": 1,
    "upper_bound": 10
  }
}'

Create a new version for a metric

post

/v1/{organization}/metric/{metric_id}/version/

Create a new version for the given metric.

Permissions:

This endpoint requires the following permissions:

Metric:ModifyMetric for the metric to create a new version for.

Authorizations

AuthorizationstringRequired

The username should be set to {org_id}_{user_id}, and the password should be the Amigo issued JWT token that identifies the user.

AuthorizationstringRequired

Amigo issued JWT token that identifies an user. It's issued either after logging in through the frontend, or manually through the SignInWithAPIKey endpoint.

X-ORG-IDstringRequired

Path parameters

organizationstringRequired

metric_idstringRequired

The ID of the metric to create a new version for.

Pattern: ^[a-f0-9]{24}$

Query parameters

versioninteger · nullableOptional

The version number of the new metric version. If specified, this endpoint throws an error if the next version of the metric in the database doesn't equal to the value of this parameter.

Header parameters

x-mongo-cluster-namestring · nullableOptional

The Mongo cluster name to perform this request in. This is usually not needed unless the organization does not exist yet in the Amigo organization infra config database.

Sec-WebSocket-Protocolstring[]OptionalDefault: []

Body

descriptionstring · min: 1Required

The description of the metric. This tells the LLM what the metric is and how the metric result should be generated.

metric_valueone ofRequired

Responses

201

Succeeded.

application/json

versionintegerRequired

Numerical version ID for the version just created. For the first version of a metric, this field will be 1.

400

The metric value type cannot be changed between versions.

401

Invalid authorization credentials.

403

Missing required permissions.

404

Specified organization or metric is not found.

409

The next version of the metric in the database doesn't match the specified version.

422

Invalid request path parameter, request body, or request query parameter failed validation.

429

The user has exceeded the rate limit of 60 requests per minute for this endpoint.

503

The service is going through temporary maintenance.

post/v1/{organization}/metric/{metric_id}/version/

POST /v1/{organization}/metric/{metric_id}/version/ HTTP/1.1
Host: api.amigo.ai
Authorization: Bearer YOUR_SECRET_TOKEN
X-ORG-ID: YOUR_API_KEY
Content-Type: application/json
Accept: */*
Content-Length: 85

{
  "description": "text",
  "metric_value": {
    "type": "text",
    "lower_bound": 1,
    "upper_bound": 1
  }
}

{
  "version": 1
}

Get Metric Versions

Retrieve the versions of a metric. The version query parameter accepts an exact version number, latest for the current version, or an inclusive range separated by - (for example, 2-5 or 1-latest). Results can be paged with limit (max 10) and continuation_token, and sorted by version.

Permission required: Metric:GetMetric

curl --request GET \
     --url 'https://api.amigo.ai/v1/<YOUR-ORG-ID>/metric/<METRIC-ID>/version?version=latest' \
     --header 'Authorization: Bearer <AUTH-TOKEN>' \
     --header 'Accept: application/json'

Get metric versions

get

/v1/{organization}/metric/{metric_id}/version

Retrieve the versions of a metric.

Permissions

This endpoint requires the following permissions:

Metric:GetMetric for the metric whose versions to retrieve.

Authorizations

AuthorizationstringRequired

The username should be set to {org_id}_{user_id}, and the password should be the Amigo issued JWT token that identifies the user.

AuthorizationstringRequired

Amigo issued JWT token that identifies an user. It's issued either after logging in through the frontend, or manually through the SignInWithAPIKey endpoint.

X-ORG-IDstringRequired

Path parameters

organizationstringRequired

metric_idstringRequired

The ID of the metric to retrieve versions for.

Pattern: ^[a-f0-9]{24}$

Query parameters

versionstring · nullableOptional

The versions of the metric to retrieve. One can specify an exact version to retrieve, which is either the version number or latest, which retrieves the latest version. Alternatively, one can specify a range of inclusive lower and upper bound for the version number separated by -, and every version within the range would be retrieved.

Example: 1

limitinteger · max: 10Optional

The maximum number of metric versions to return.

Default: 10

continuation_tokenintegerOptional

The continuation token from the previous request used to retrieve the next page of metric versions.

Default: 0

sort_bystring[]Optional

The fields to sort the versions by. Supported fields are version. Specify a + before the field name to indicate ascending sorting and - for descending sorting. Multiple fields can be specified to break ties.

Default: []

Header parameters

x-mongo-cluster-namestring · nullableOptional

The Mongo cluster name to perform this request in. This is usually not needed unless the organization does not exist yet in the Amigo organization infra config database.

Sec-WebSocket-Protocolstring[]OptionalDefault: []

Responses

200

Succeeded.

application/json

has_morebooleanRequired

Whether there are more metric versions that can be retrieved with further pagination.

continuation_tokeninteger · nullableRequired

The continuation token to be used to retrieve the next page of metric versions. This is only included if has_more is true.

401

Invalid authorization credentials.

403

Missing required permissions.

404

Specified organization or metric is not found.

422

Invalid request path parameter or request query parameter failed validation.

429

The user has exceeded the rate limit of 60 requests per minute for this endpoint.

503

The service is going through temporary maintenance.

get/v1/{organization}/metric/{metric_id}/version

GET /v1/{organization}/metric/{metric_id}/version HTTP/1.1
Host: api.amigo.ai
Authorization: Bearer YOUR_SECRET_TOKEN
X-ORG-ID: YOUR_API_KEY
Accept: */*

{
  "metric_versions": [
    {
      "metric_id": "text",
      "version": 1,
      "description": "text",
      "metric_value": {
        "type": "text"
      },
      "creator": {
        "org_id": "text",
        "user_id": "text"
      },
      "created_at": "2026-01-01T00:00:00.000Z"
    }
  ],
  "has_more": true,
  "continuation_token": 1
}

Delete a Metric

Delete a metric. The metric is no longer modifiable or manually evaluable, but remains stored, and existing services or simulations that evaluate this metric continue to do so. You can still retrieve it by ID to view its specifics.

Deletion will fail with HTTP 400 if the metric is currently used in any simulation unit tests. Remove the metric from all simulation unit tests before deleting.

Delete a metric

delete

/v1/{organization}/metric/{metric_id}/

Delete a metric. The metric is no longer modifiable or manually evaluable, but remains stored, and existing services or simulations that evaluate this metric will continue to do so. One can still retrieve it using its ID to view its specifics.

Permissions

This endpoint requires the following permissions:

Metric:DeleteMetric for the metric to delete.

Authorizations

AuthorizationstringRequired

The username should be set to {org_id}_{user_id}, and the password should be the Amigo issued JWT token that identifies the user.

AuthorizationstringRequired

Amigo issued JWT token that identifies an user. It's issued either after logging in through the frontend, or manually through the SignInWithAPIKey endpoint.

X-ORG-IDstringRequired

Path parameters

organizationstringRequired

metric_idstringRequired

The ID of the metric to update.

Pattern: ^[a-f0-9]{24}$

Header parameters

x-mongo-cluster-namestring · nullableOptional

The Mongo cluster name to perform this request in. This is usually not needed unless the organization does not exist yet in the Amigo organization infra config database.

Sec-WebSocket-Protocolstring[]OptionalDefault: []

Responses

200

Succeeded.

application/json

anyOptional

400

Metric is used in a simulation unit test.

401

Invalid authorization credentials.

403

Missing required permissions.

404

Specified organization or metric is not found.

422

Invalid request path parameter failed validation.

429

The user has exceeded the rate limit of 100 requests per minute for this endpoint.

503

The service is going through temporary maintenance.

delete/v1/{organization}/metric/{metric_id}/

DELETE /v1/{organization}/metric/{metric_id}/ HTTP/1.1
Host: api.amigo.ai
Authorization: Bearer YOUR_SECRET_TOKEN
X-ORG-ID: YOUR_API_KEY
Accept: */*

No content

Evaluate Metrics

Evaluate one or more metrics against a completed conversation. Results are stored and retrievable through the Get Evaluation Results endpoint with a source type of manual.

If the same conversation has been manually evaluated for the same metric previously, the previous evaluation result is overwritten.

curl --request POST \
     --url 'https://api.amigo.ai/v1/<YOUR-ORG-ID>/metric/evaluate' \
     --header 'Authorization: Bearer <AUTH-TOKEN>' \
     --header 'Accept: application/json' \
     --header 'Content-Type: application/json' \
     --data '{
  "metric_ids": ["6618791275130b73714e8d1c", "6618791275130b73714e8d1d"],
  "conversation_id": "6618791275130b73714eaaaa"
}'

from amigo_sdk import AmigoClient

with AmigoClient() as client:
    result = client.metric.evaluate(
        metric_ids=["6618791275130b73714e8d1c"],
        conversation_id="6618791275130b73714eaaaa",
    )
    for evaluation in result.metrics:
        print(f"Metric: {evaluation.metric_id}, Result: {evaluation.result}")

import { AmigoClient } from "@amigo-ai/sdk";

const client = new AmigoClient({ ...config });
const result = await client.metric.evaluate({
  metric_ids: ["6618791275130b73714e8d1c"],
  conversation_id: "6618791275130b73714eaaaa",
});
result.metrics?.forEach((evaluation) => {
  console.log(`Metric: ${evaluation.metric_id}, Result: ${evaluation.result}`);
});

Request body:

Field

Type

Required

Description

metric_ids

array<string>

Yes

IDs of metrics to evaluate (1-10, must be unique)

conversation_id

string

Yes

ID of the completed conversation to evaluate

evaluate_to_interaction_id

string

If specified, only messages up to (and including) this interaction are evaluated

The conversation must have at least one interaction, or the request returns HTTP 400.

Evaluate metrics

post

/v1/{organization}/metric/evaluate

Evaluate the latest versions of specified metrics for a completed conversation. The results will be stored and retrievable through the GetMetricEvaluationResults endpoint with a special source type of manual.

If the same conversation has been manually evaluated for the same metric, the previous evaluation result will be overwritten.

Permissions

This endpoint requires the following permissions:

Metric:GetMetric for the metrics.
Metric:EvaluateMetric for the metrics.
Metric:GetMetricEvaluationResult for the metric results.

Authorizations

AuthorizationstringRequired

The username should be set to {org_id}_{user_id}, and the password should be the Amigo issued JWT token that identifies the user.

AuthorizationstringRequired

Amigo issued JWT token that identifies an user. It's issued either after logging in through the frontend, or manually through the SignInWithAPIKey endpoint.

X-ORG-IDstringRequired

Path parameters

organizationstringRequired

Header parameters

x-mongo-cluster-namestring · nullableOptional

The Mongo cluster name to perform this request in. This is usually not needed unless the organization does not exist yet in the Amigo organization infra config database.

Sec-WebSocket-Protocolstring[]OptionalDefault: []

Body

metric_idsstring[] · min: 1 · max: 10Required

The IDs of the metrics to evaluate.

conversation_idstringRequired

The ID of the conversation to evaluate the metrics for.

Pattern: ^[a-f0-9]{24}$

evaluate_to_interaction_idstring · nullableOptional

If specified, only messages up to (and including) this interaction will be evaluated.

Pattern: ^[a-f0-9]{24}$

Responses

200

Succeeded.

application/json

400

The conversation has no interactions.

401

Invalid authorization credentials.

403

Missing required permissions.

404

Specified organization, conversation, or metric is not found.

422

Invalid request path parameter or request body failed validation.

429

The user has exceeded the rate limit of 50 requests per minute for this endpoint.

503

The service is going through temporary maintenance.

post/v1/{organization}/metric/evaluate

POST /v1/{organization}/metric/evaluate HTTP/1.1
Host: api.amigo.ai
Authorization: Bearer YOUR_SECRET_TOKEN
X-ORG-ID: YOUR_API_KEY
Content-Type: application/json
Accept: */*
Content-Length: 84

{
  "metric_ids": [
    "text"
  ],
  "conversation_id": "text",
  "evaluate_to_interaction_id": "text"
}

{
  "metrics": [
    {
      "metric_id": "text",
      "value": 1,
      "references": [
        "text"
      ],
      "justification": "text"
    }
  ]
}

Get Evaluation Results

Retrieve metric evaluation results matching the given filters. Results include the metric ID, evaluated value, source type, and associated conversation or simulation run.

Common filters:

metric_id=<id> (repeatable, deprecated): filter by metric
metric_id_and_version=<metric_id>-<version> (repeatable): filter by metric and version; leave <version> empty to match any version
source_type=post-session|manual|simulation (repeatable): filter by evaluation source
conversation_id=<id> (repeatable): filter by conversation (for post-session and manual sources)
simulation_unit_test_set_run_id=<id> (repeatable): filter by simulation run (for simulation source)
timestamp_after=<ISO8601>: only results evaluated on or after this time
timestamp_before=<ISO8601>: only results evaluated on or before this time
sort_by=+created_at|-created_at (repeatable)
limit (0-500, default 500), continuation_token (int, default 0)

curl --request GET \
     --url 'https://api.amigo.ai/v1/<YOUR-ORG-ID>/metric/metric_evaluation_result?metric_id=6618791275130b73714e8d1c&source_type=manual&limit=20' \
     --header 'Authorization: Bearer <AUTH-TOKEN>' \
     --header 'Accept: application/json'

from amigo_sdk import AmigoClient

with AmigoClient() as client:
    result = client.metric.get_evaluation_results(
        metric_id=["6618791275130b73714e8d1c"],
        source_type=["manual"],
        limit=20,
    )
    for r in result.metric_evaluation_results:
        print(f"Metric: {r.metric_id}, Result: {r.result}, Source: {r.source_type}")

import { AmigoClient } from "@amigo-ai/sdk";

const client = new AmigoClient({ ...config });
const result = await client.metric.getEvaluationResults({
  metric_id: ["6618791275130b73714e8d1c"],
  source_type: ["manual"],
  limit: 20,
});
result.metric_evaluation_results?.forEach((r) => {
  console.log(`Metric: ${r.metric_id}, Result: ${r.result}`);
});

Get metric evaluation results

get

/v1/{organization}/metric/metric_evaluation_result

Retrieve metric evaluation results that match the given filters.

Permissions

This endpoint may be impacted by the following permissions:

Only metric evaluation results that the authenticated user has Metric:GetMetricEvaluationResult permission for will be retrieved.

Authorizations

AuthorizationstringRequired

The username should be set to {org_id}_{user_id}, and the password should be the Amigo issued JWT token that identifies the user.

AuthorizationstringRequired

Amigo issued JWT token that identifies an user. It's issued either after logging in through the frontend, or manually through the SignInWithAPIKey endpoint.

X-ORG-IDstringRequired

Path parameters

organizationstringRequired

Query parameters

limitinteger · max: 500Optional

The maximum number of metrics to return.

Default: 500

continuation_tokenintegerOptional

The continuation token from the previous request used to retrieve the next page of metrics.

Default: 0

timestamp_afterstring · date-time · nullableOptional

An ISO8601 timestamp. Only results evaluated on or after this moment is returned.

timestamp_beforestring · date-time · nullableOptional

An ISO8601 timestamp. Only results evaluated on or before this moment is returned.

metric_idstring[]OptionalDeprecated

The ID of the metric to retrieve.

Default: []

metric_id_and_versionstring[]Optional

The metric IDs and their versions to retrieve. Each value must be in the format of <metric_id>-<version>. If version is left empty, any versions of the metric would be retrieved.

Default: []

conversation_idstring[]Optional

For metric evaluation sources with type post-session or manual, this field filters the metric evaluation results to only include those that were computed from the given conversation IDs.

Default: []

simulation_unit_test_set_run_idstring[]Optional

For metric evaluation sources with type simulation, this field filters the metric evaluation results to only include those that were computed from the given simulation unit test set run IDs.

Default: []

sort_bystring[]Optional

The fields to sort the metrics by. Supported fields are created_at. Specify a + before the field name to indicate ascending sorting and - for descending sorting. Multiple fields can be specified to break ties.

Default: []

Header parameters

x-mongo-cluster-namestring · nullableOptional

The Mongo cluster name to perform this request in. This is usually not needed unless the organization does not exist yet in the Amigo organization infra config database.

Sec-WebSocket-Protocolstring[]OptionalDefault: []

Responses

200

Succeeded

application/json

has_morebooleanRequired

Whether there are more metric evaluation results to retrieve.

continuation_tokenany · nullableRequired

The continuation token to use to retrieve the next set of metric evaluation results.

401

Invalid authorization credentials.

403

Missing required permissions.

404

Specified organization is not found.

422

Invalid request path parameter or request query parameter failed validation.

429

The user has exceeded the rate limit of 10 requests per minute for this endpoint.

503

The service is going through temporary maintenance.

get/v1/{organization}/metric/metric_evaluation_result

GET /v1/{organization}/metric/metric_evaluation_result HTTP/1.1
Host: api.amigo.ai
Authorization: Bearer YOUR_SECRET_TOKEN
X-ORG-ID: YOUR_API_KEY
Accept: */*

{
  "metric_evaluation_results": [
    {
      "id": "text",
      "org_id": "text",
      "created_at": "2026-01-01T00:00:00.000Z",
      "updated_at": "2026-01-01T00:00:00.000Z",
      "metric_id": "text",
      "metric_version": 1,
      "result": 1,
      "justification": "text",
      "source": {
        "type": "post-session",
        "conversation_id": "text",
        "references": [
          "text"
        ]
      }
    }
  ],
  "has_more": true,
  "continuation_token": null,
  "filter_values": {
    "metric_ids": [
      "text"
    ]
  }
}

Data Access → Metric Tables
Data Access → Simulation Tables
Classic API -> Services (metric application through services)
Getting Started → Authentication

PreviousDynamic Behaviors NextSimulations

Last updated 25 days ago

Was this helpful?

Good morning

Overview

Metric Value Types

Evaluation Sources

Metric Versioning

Typical Lifecycle

Rate Limits

Create a Metric

Create a metric

Permissions

List Metrics

Get metrics

Permissions

Search Metrics

Search metrics

Permissions

Update a Metric

Update a metric

Permissions

Create a Metric Version

Create a new version for a metric

Permissions:

Get Metric Versions

Get metric versions

Permissions

Delete a Metric

Delete a metric

Permissions

Evaluate Metrics

Evaluate metrics

Permissions

Get Evaluation Results

Get metric evaluation results

Permissions

Related