Unit Tests
Combine a persona, scenario, service, and success criteria into a single repeatable test case.
Create a Unit Test
curl --request POST \
--url 'https://api.amigo.ai/v1/<YOUR-ORG-ID>/simulation/unit_test/' \
--header 'Authorization: Bearer <AUTH-TOKEN>' \
--header 'Content-Type: application/json' \
--data '{
"name": "Onboarding - Confused New User",
"description": "Tests that the agent successfully guides a confused new user through account setup.",
"service_id": "6618791275130b73714e8d1c",
"service_version_set_name": "release",
"persona_id": "6618791275130b73714e8d2a",
"scenario_id": "6618791275130b73714e8d3b",
"max_interactions": 20,
"success_criterions": [
{
"name": "Completion Rate",
"metric_id": "6618791275130b73714e8d4c",
"criterion": {
"type": "boolean",
"expected_value": true
}
}
],
"tags": {"suite": "onboarding", "priority": "high"}
}'Success Criteria
Create a new simulation unit test.
Permissions
This endpoint requires the following permissions:
Simulation:CreateSimulationUnitTestfor the unit test to create.
The username should be set to {org_id}_{user_id}, and the password should be the Amigo issued JWT token that identifies the user.
Amigo issued JWT token that identifies an user. It's issued either after logging in through the frontend, or manually through the SignInWithAPIKey endpoint.
An optional organization identifier that indicates from which organization the token is issued. This is used in rare cases where the user to authenticate is making a request for resources in another organization.
The Mongo cluster name to perform this request in. This is usually not needed unless the organization does not exist yet in the Amigo organization infra config database.
[]The name of the unit test.
A description of the unit test.
The service that the unit test runs on.
^[a-f0-9]{24}$The version set of the service that the unit test runs on.
The persona that unit test uses.
^[a-f0-9]{24}$The scenario that the unit test uses.
^[a-f0-9]{24}$The max number of interactions after which the unit test fails.
Succeeded.
The metric type of the success criterion does not match the metric type of the metric.
Invalid authorization credentials.
Missing required permissions.
The specified organization, service, persona, scenario, metric, or service version set was not found.
A unit test with this name already exists.
Invalid request path parameter or request body failed validation.
The user has exceeded the rate limit of 500 requests per minute for this endpoint.
The service is going through temporary maintenance.
List Unit Tests
Retrieve the simulation unit tests that match the given filters.
Permissions
This endpoint may be impacted by the following permissions:
- Only simulation unit tests that the authenticated user has
Simulation:GetSimulationUnitTestpermission for will be retrieved.
The username should be set to {org_id}_{user_id}, and the password should be the Amigo issued JWT token that identifies the user.
Amigo issued JWT token that identifies an user. It's issued either after logging in through the frontend, or manually through the SignInWithAPIKey endpoint.
An optional organization identifier that indicates from which organization the token is issued. This is used in rare cases where the user to authenticate is making a request for resources in another organization.
The IDs of the simulation unit tests to retrieve.
[]The maximum number of simulation unit tests to return.
50The continuation token from the previous request used to retrieve the next page of simulation unit tests.
0Whether the simulation unit test is deleted.
The IDs of the services that the simulation unit tests run on.
[]The IDs of the personas that the simulation unit tests use.
[]The IDs of the scenarios that the simulation unit tests use.
[]The IDs of the metrics that the simulation unit tests use.
[]The creators of the simulation unit tests. Each value must be of the format org_id,user_id.
[]The tags of the simulation unit tests. Must be specified using the syntax key:value, which means to match all sets with the given key and value pair among its tags. If value is *, it means the value does not matter. If value is empty, it matches against when the value is None.
[]The fields to sort the sets by. Supported fields are updated_at. Specify a + before the field name to indicate ascending sorting and - for descending sorting. Multiple fields can be specified to break ties.
[]The Mongo cluster name to perform this request in. This is usually not needed unless the organization does not exist yet in the Amigo organization infra config database.
[]Succeeded.
Whether there are more simulation unit tests to retrieve.
The continuation token to use to retrieve the next page of simulation unit tests.
For each filter that this endpoint supports that can take on dynamic values, this field includes what these values are. This is only provided for the first page in the pagination results.
Note that the values are counted assuming the authenticated user has access to all the simulation unit tests, so they might differ from how many sets are actually retrieved.
Invalid authorization credentials.
Missing required permissions.
Specified organization is not found.
Invalid request path parameter or request query parameter failed validation.
The user has exceeded the rate limit of 200 requests per minute for this endpoint.
The service is going through temporary maintenance.
Search Unit Tests
Search for simulation unit tests that match the given filters and whose name or description contains the specified query.
Permissions
This endpoint may be impacted by the following permissions:
- Only simulation unit tests that the authenticated user has
Simulation:GetSimulationUnitTestpermission for will be retrieved.
The username should be set to {org_id}_{user_id}, and the password should be the Amigo issued JWT token that identifies the user.
Amigo issued JWT token that identifies an user. It's issued either after logging in through the frontend, or manually through the SignInWithAPIKey endpoint.
An optional organization identifier that indicates from which organization the token is issued. This is used in rare cases where the user to authenticate is making a request for resources in another organization.
The query to search for. Any simulation unit tests containing the terms in its name would be returned.
The creators of the simulation unit tests. Each value must be of the format org_id,user_id.
[]The tags of the simulation unit tests. Must be specified using the syntax key:value, which means to match all sets with the given key and value pair among its tags. If value is *, it means the value does not matter. If value is empty, it matches against when the value is None.
[]The IDs of the services that the simulation unit tests run on.
[]The IDs of the personas that the simulation unit tests use.
[]The IDs of the scenarios that the simulation unit tests use.
[]The IDs of the metrics that the simulation unit tests use.
[]The Mongo cluster name to perform this request in. This is usually not needed unless the organization does not exist yet in the Amigo organization infra config database.
[]Succeeded.
Invalid authorization credentials.
Missing required permissions.
Specified organization is not found.
Invalid request path parameter or request query parameter failed validation.
The user has exceeded the rate limit of 50 requests per minute for this endpoint.
The service is going through temporary maintenance.
Update a Unit Test
Modify the specified simulation unit test.
Permissions
Simulation:ModifySimulationUnitTestfor the unit test to update.
The username should be set to {org_id}_{user_id}, and the password should be the Amigo issued JWT token that identifies the user.
Amigo issued JWT token that identifies an user. It's issued either after logging in through the frontend, or manually through the SignInWithAPIKey endpoint.
An optional organization identifier that indicates from which organization the token is issued. This is used in rare cases where the user to authenticate is making a request for resources in another organization.
The ID of the simulation unit test to update.
^[a-f0-9]{24}$The Mongo cluster name to perform this request in. This is usually not needed unless the organization does not exist yet in the Amigo organization infra config database.
[]A description of the unit test.
The service that the unit test runs on.
^[a-f0-9]{24}$The version set of the service that the unit test runs on.
The persona that unit test uses.
^[a-f0-9]{24}$The scenario that the unit test uses.
^[a-f0-9]{24}$The max number of interactions after which the unit test fails.
The success criterions for the unit test.
Each time this unit test is run, the number of sessions to simulate.
The tags of the simulation persona.
Succeeded.
The metric type of the success criterion does not match the metric type of the metric.
Invalid authorization credentials.
Missing required permissions.
The specified organization, simulation unit test, service, persona, scenario, metric, or service version set was not found.
Invalid request path parameter or request body failed validation.
The user has exceeded the rate limit of 500 requests per minute for this endpoint.
The service is going through temporary maintenance.
No content
Delete a Unit Test
Delete a simulation unit test
Delete a simulation unit test. The unit test is soft-deleted and all references to it are still valid. Any scheduled unit test runs would still continue as expected, but no new run can be created.
This endpoint will error if the simulation unit test is used in any simulation unit test set.
Permissions
This endpoint requires the following permissions:
Simulation:DeleteSimulationUnitTestfor the simulation unit test to delete.
The username should be set to {org_id}_{user_id}, and the password should be the Amigo issued JWT token that identifies the user.
Amigo issued JWT token that identifies an user. It's issued either after logging in through the frontend, or manually through the SignInWithAPIKey endpoint.
An optional organization identifier that indicates from which organization the token is issued. This is used in rare cases where the user to authenticate is making a request for resources in another organization.
The ID of the simulation unit test to delete.
^[a-f0-9]{24}$The Mongo cluster name to perform this request in. This is usually not needed unless the organization does not exist yet in the Amigo organization infra config database.
[]Succeeded.
The specified simulation unit test is used in a simulation unit test set.
Invalid authorization credentials.
Missing required permissions.
The specified organization or simulation unit test was not found or is inactive.
Invalid request path parameter failed validation.
The user has exceeded the rate limit of 500 requests per minute for this endpoint.
The service is going through temporary maintenance.
No content
Related
Last updated
Was this helpful?