Unit Test Set Runs
Execute test suites, monitor progress, cancel runs, and download result artifacts.
Create a Run
curl --request POST \
--url 'https://api.amigo.ai/v1/<YOUR-ORG-ID>/simulation/unit_test_set_run/' \
--header 'Authorization: Bearer <AUTH-TOKEN>' \
--header 'Content-Type: application/json' \
--data '{
"simulation_unit_test_set_id": "6618791275130b73714e8d7f"
}'{
"simulation_unit_test_set_run_id": "6618791275130b73714e8d9h"
}Create a run for a simulation unit test set. The user must not already have an unfinished run for this simulation unit test set.
Permissions
This endpoint requires the following permissions:
Simulation:CreateSimulationUnitTestSetRunfor the test set run.
The username should be set to {org_id}_{user_id}, and the password should be the Amigo issued JWT token that identifies the user.
Amigo issued JWT token that identifies an user. It's issued either after logging in through the frontend, or manually through the SignInWithAPIKey endpoint.
An optional organization identifier that indicates from which organization the token is issued. This is used in rare cases where the user to authenticate is making a request for resources in another organization.
The Mongo cluster name to perform this request in. This is usually not needed unless the organization does not exist yet in the Amigo organization infra config database.
[]The ID of the simulation unit test set to run.
^[a-f0-9]{24}$Succeeded.
Invalid authorization credentials.
Missing required permissions.
The specified organization or simulation unit test set was not found.
The user already has an unfinished run for this simulation unit test set.
Invalid request path parameter or request body failed validation.
The user has exceeded the rate limit of 20 requests per minute for this endpoint.
The service is going through temporary maintenance.
List Runs
Retrieves simulation unit test set runs.
This endpoint may be impacted by the following permissions:
- Only the runs that the authenticated user has the
Simulation:GetSimulationUnitTestSetRunpermission for will be returned. - The
unit_test_run_specsandunit_test_run_resultsfields are only populated if the authenticated user has theSimulation:GetSimulationUnitTestSetpermission. - The fields
scenario_version_info,persona_version_info,metrics_to_evaluateinunit_test_run_specsand the fieldmetric_evaluation_resultsinunit_test_run_resultsare only populated if the authenticated user has theSimulation:GetSimulationUnitTestpermission.
The username should be set to {org_id}_{user_id}, and the password should be the Amigo issued JWT token that identifies the user.
Amigo issued JWT token that identifies an user. It's issued either after logging in through the frontend, or manually through the SignInWithAPIKey endpoint.
An optional organization identifier that indicates from which organization the token is issued. This is used in rare cases where the user to authenticate is making a request for resources in another organization.
The ID of the simulation unit test set run to retrieve.
[]The maximum number of simulation unit test set runs to return.
50The continuation token from the previous request used to retrieve the next page of simulation unit test set runs.
0The creators of the simulation unit test set runs to retrieve. Each value must be of the format org_id,user_id.
[]The fields to sort the sets by. Supported fields are created_at.
[]The IDs of the simulation unit test sets that the simulation unit test set runs belong to.
[]Whether the simulation unit test set run is completed.
Whether the simulation unit test set run has failed due to failing metrics. Note that this field could be True even if the run is still ongoing -- it simply means some unit test runs have failed.
Whether the simulation unit test set run has encountered an error.
Filter for simulation unit test set runs created at or after this time (inclusive).
Filter for simulation unit test set runs created at or before this time (inclusive).
The Mongo cluster name to perform this request in. This is usually not needed unless the organization does not exist yet in the Amigo organization infra config database.
[]Succeeded.
Whether there are more simulation unit test set runs to retrieve.
The continuation token to use to retrieve the next page of simulation unit test set runs.
For each filter that this endpoint supports that can take on dynamic values, this field includes what these values are. This is only provided for the first page in the pagination results.
Note that the values are counted assuming the authenticated user has access to all the simulation unit test set runs, so they might differ from how many sets are actually retrieved.
Invalid authorization credentials.
Missing required permissions.
Specified organization is not found.
Invalid request path parameter or request query parameter failed validation.
The user has exceeded the rate limit of 50 requests per minute for this endpoint.
The service is going through temporary maintenance.
Cancel a Run
Cancel a simulation unit test set run
Cancel a simulation unit test set run. The run must still be in progress. This cancellation is a best-effort operation, and there's no guarantee that it'll take effect immediately.
Permissions
This endpoint requires the following permissions:
Simulation:CancelSimulationUnitTestSetRunfor the authenticated user.
The username should be set to {org_id}_{user_id}, and the password should be the Amigo issued JWT token that identifies the user.
Amigo issued JWT token that identifies an user. It's issued either after logging in through the frontend, or manually through the SignInWithAPIKey endpoint.
An optional organization identifier that indicates from which organization the token is issued. This is used in rare cases where the user to authenticate is making a request for resources in another organization.
The ID of the simulation unit test set run to cancel.
^[a-f0-9]{24}$The Mongo cluster name to perform this request in. This is usually not needed unless the organization does not exist yet in the Amigo organization infra config database.
[]Succeeded.
Invalid authorization credentials.
Missing required permissions.
The specified organization or simulation unit test set run was not found.
The simulation unit test set run is not in progress.
Invalid request path parameter failed validation.
The user has exceeded the rate limit of 10 requests per minute for this endpoint.
The service is going through temporary maintenance.
No content
Get Run Artifacts
Get simulation unit test set run artifacts
Get the artifacts of a simulation unit test set run. A link to the raw artifacts in S3 is returned.
Permissions
This endpoint requires the following permissions:
Simulation:GetSimulationUnitTestSetRunfor the simulation unit test set run.Organization:GetServiceHierarchicalStateMachinefor the state machine that the simulation uses.
The username should be set to {org_id}_{user_id}, and the password should be the Amigo issued JWT token that identifies the user.
Amigo issued JWT token that identifies an user. It's issued either after logging in through the frontend, or manually through the SignInWithAPIKey endpoint.
An optional organization identifier that indicates from which organization the token is issued. This is used in rare cases where the user to authenticate is making a request for resources in another organization.
The ID of the simulation unit test set run to retrieve the artifacts of.
^[a-f0-9]{24}$The Mongo cluster name to perform this request in. This is usually not needed unless the organization does not exist yet in the Amigo organization infra config database.
[]Succeeded.
An URL where the artifacts can be downloaded from.
The time in UTC when the presigned URL expires.
The artifacts for the unit test set run are not yet available for retrieval
Invalid authorization credentials.
Missing required permissions.
Specified organization or unit test set run is not found.
Invalid request path parameter failed validation.
The user has exceeded the rate limit of 100 requests per minute for this endpoint.
The service is going through temporary maintenance.
Run Lifecycle
Related
Last updated
Was this helpful?