API Reference

Evaluation

GET /v1/evaluations

Get Evaluations

Query Parameters:

  • user_email ({'null', 'string'})

  • status ({'array', 'null'})

  • limit (integer)

  • offset (integer)

  • sort_by (string)

  • sort_order (string)

  • name ({'null', 'string'})

  • hub ({'null', 'string'})

  • model ({'null', 'string'})

  • created_at ({'integer', 'null'})

  • completed_at ({'integer', 'null'})

  • tags ({'array', 'null'})

Status Codes:
POST /v1/evaluations

Create Evaluation

Status Codes:
GET /v1/evaluations/{evaluation_id}/response-errors

Get Response Errors

Parameters:

  • evaluation_id (string)

Query Parameters:

  • limit (integer) – Maximum number of error types to return

Status Codes:
GET /v1/evaluations/{id}

Get Evaluation

Parameters:

  • id (string)

Status Codes:
PUT /v1/evaluations/{id}

Update Evaluation Name

Parameters:

  • id (string)

Status Codes:
GET /v1/evaluations/metadata/{id}

Get Evaluation With Metadata

Parameters:

  • id (string)

Status Codes:
POST /v1/evaluations/{id}/score

Calculate Evaluation Score

Parameters:

  • id (string)

Status Codes:
POST /v1/evaluations/{id}/delete

Delete Evaluation

Parameters:

  • id (string)

Status Codes:
POST /v1/evaluations/{id}/cancel

Cancel Evaluation

Parameters:

  • id (string)

Status Codes:
POST /v1/evaluations/{id}/reports

Create Evaluation Report

Parameters:

  • id (string)

Status Codes:
GET /v1/evaluations/{id}/reports/{report_id}

Get Evaluation Report

Parameters:

  • id (string)

  • report_id (string)

Status Codes:
DELETE /v1/evaluations/{id}/reports/{report_id}

Delete Evaluation Report

Parameters:

  • id (string)

  • report_id (string)

Status Codes:
GET /v1/evaluations/{id}/list-reports

Get Report Ids For Evaluation

Parameters:

  • id (string)

Query Parameters:

  • status ({'null', 'string'})

Status Codes:
POST /v1/evaluation-estimate

Calculate Evaluation Estimate

Status Codes:

Harness

GET /v1/harness

Get Harnesses

Query Parameters:

  • evaluation_id (string) – (Required)

  • ids ({'array', 'null'})

Status Codes:
GET /v1/harness/{id}

Get Harness By Id

Parameters:

  • id (string)

Status Codes:
POST /v1/harness-creation-agent/chat

Post Harness Creation Chat

Forwards request to the harness_creation_domain to get a response or trigger a new harness creation

Status Codes:
POST /v1/harness-creation-agent

Custom Harness Creation

Status Codes:
POST /v1/harness-configs

Create Harness

Creates a new harness configuration for the current user and team. - Validates write permissions and user authentication. - Retrieves the current user from the authentication token. - Constructs the harness configuration data from the request. - Delegates creation to the harness_creation_domain. Returns the created HarnessConfig object.

Status Codes:
GET /v1/harness-configs

Get Harness Configs

List harness configs, optionally filtered and sorted. sort_by: Field to sort by. Allowed: ‘created_at’, ‘name’, ‘id’.

Query Parameters:

  • harness_type ({'array', 'null'})

  • with_stages ({'null', 'boolean'})

  • user_id ({'null', 'string'})

  • limit ({'integer', 'null'})

  • offset ({'integer', 'null'})

  • version ({'null', 'string'})

  • sort_by (string)

  • sort_order (string)

Status Codes:
GET /v1/harness-configs/versions

Get Harness Config Versions

Status Codes:

  • 200 OK – Successful Response

GET /v1/harness-configs/{harness_config_id}

Get Harness Config By Id

Parameters:

  • harness_config_id (string)

Query Parameters:

  • version (string) – (Required)

  • with_stages ({'null', 'boolean'})

Status Codes:
DELETE /v1/harness-configs/{harness_config_id}

Delete Harness Config

Parameters:

  • harness_config_id (string)

Query Parameters:

  • version (string) – (Required)

Status Codes:
PUT /v1/harness-configs/{harness_config_id}/versions/{version}/archive

Archive Harness Config

Parameters:

  • harness_config_id (string)

  • version (string)

Status Codes:
GET /v1/harness-configs/custom/eval-config-inputs

Get Eval Config Inputs

Returns a model_dump of the CustomEvalConfigInputs class

Status Codes:

  • 200 OK – Successful Response

POST /v1/harness-configs/{harness_config_id}/stages

Create

Parameters:

  • harness_config_id (string)

Status Codes:
GET /v1/harness-configs/{harness_config_id}/stages

Get Stages For Harness Config

Get all stages for a harness config

Parameters:

  • harness_config_id (string)

Status Codes:
GET /v1/harness-configs/{harness_config_id}/stages/{staging_id}

Get Stage By Id

Get a stage by ID

Parameters:

  • harness_config_id (string)

  • staging_id (string)

Status Codes:
PUT /v1/harness-configs/{harness_config_id}/stages/{staging_id}

Update Stage

Parameters:

  • staging_id (string)

Status Codes:
DELETE /v1/harness-configs/{harness_config_id}/stages/{staging_id}

Delete Stage

Delete a stage by ID

Parameters:

  • staging_id (string)

Status Codes:
POST /v1/harness-configs/{harness_config_id}/stages/{staging_id}/files

Request File Upload

Request a file upload URL for a stage

Parameters:

  • harness_config_id (string)

  • staging_id (string)

Status Codes:
GET /v1/harness-configs/{harness_config_id}/stages/{staging_id}/files

Get Stage Files

Get all files associated with a stage

Parameters:

  • harness_config_id (string)

  • staging_id (string)

Status Codes:
DELETE /v1/harness-configs/{harness_config_id}/stages/{staging_id}/files/{file_id}

Delete Stage File

Delete a file from a stage

Parameters:

  • harness_config_id (string)

  • staging_id (string)

  • file_id (string)

Status Codes:

Scenario

GET /v1/scenarios

Get Scenarios

Query Parameters:

  • evaluation_id (string) – (Required)

  • ids ({'array', 'null'})

Status Codes:
GET /v1/scenarios/{id}

Get Scenario By Id

Parameters:

  • id (string)

Status Codes:
GET /v1/scenario-configs

Get Scenario Config By Ids

Query Parameters:

  • ids (array)

  • version (string)

Status Codes:

Probe

GET /v1/probes

Get Probes

Query Parameters:

  • evaluation_id (string) – (Required)

  • ids ({'array', 'null'})

Status Codes:
GET /v1/probes/{id}

Get Probe By Id

Parameters:

  • id (string)

Status Codes:
GET /v1/probe-configs

Get Probe Configs

Query Parameters:

  • ids (array)

  • version (string)

  • limit (integer)

  • offset (integer)

Status Codes:
PUT /v1/probe-configs/{id}

Update Probe Configs

Parameters:

  • id (string)

Status Codes:

Prompt

POST /v1/prompts

Create Prompt

Status Codes:
GET /v1/prompts

Get Prompts

Query Parameters:

  • ids (array)

  • limit (integer)

  • offset (integer)

Status Codes:
PUT /v1/prompts/{id}

Update Prompt

Parameters:

  • id (string)

Status Codes:
GET /v1/prompts/{id}

Get Prompt By Id

Query Parameters:

  • prompt_id (string) – (Required)

  • team_id (string) – (Required)

Status Codes:
GET /v1/prompts/search/find

Search Prompts

Query Parameters:

  • query (string) – (Required)

  • limit (integer)

  • offset (integer)

  • nearest_neighbors (integer)

  • total_limit (integer)

Status Codes:

Response

GET /v1/responses/{id}

Get Response By Id

Parameters:

  • id (string)

Status Codes:
GET /v1/responses

Get Responses

Query Parameters:

  • evaluation_id ({'null', 'string'})

  • status ({'array', 'null'})

  • ids ({'array', 'null'})

  • probe_ids ({'array', 'null'})

  • prompt_ids ({'array', 'null'})

  • is_visible ({'null', 'string'})

  • order_by ({'null', 'string'})

  • limit (integer) – The maximum number of items to return

  • offset (integer) – The number of items to skip before starting to collect the result set

Status Codes:

Detector

GET /v1/detectors

Get Detectors

Query Parameters:

  • version ({'null', 'string'}) – (Required)

  • ids (array)

Status Codes:
GET /v1/detectors/{id}

Get Detector By Id

Parameters:

  • id (string)

Query Parameters:

  • version ({'null', 'string'}) – (Required)

Status Codes:
GET /v1/detections/{id}

Get Detection By Id

Parameters:

  • id (string)

Status Codes:
POST /v1/detections

Create Detection

Status Codes:

API Key

POST /v1/api-keys

Create Api Key

Status Codes:
GET /v1/api-keys

Get Api Keys

Query Parameters:

  • hubs ({'null', 'string'})

Status Codes:
POST /v1/api-keys/test

Test Api Key

Status Codes:
PUT /v1/api-keys/{id}

Update Api Key

Parameters:

  • id (string)

Status Codes:
DELETE /v1/api-keys/{id}

Delete Api Key

Parameters:

  • id (string)

Status Codes:
GET /v1/api-keys/{id}

Get Api Key By Id

Parameters:

  • id (string)

Status Codes:
GET /v1/api-keys/{id}/models

List Models For Key

Parameters:

  • id (string)

Status Codes:

Hub

GET /v1/hubs

Get Hubs

Status Codes:

  • 200 OK – Successful Response

POST /v1/hubs/cost

Compute Cost

Status Codes:

Test

GET /v1/tests/{id}

Get Test By Id

Parameters:

  • id (string)

Status Codes:
GET /v1/tests

Get Tests

Query Parameters:

  • evaluation_id ({'null', 'string'})

  • ids ({'array', 'null'})

  • probe_ids ({'array', 'null'})

  • status ({'null', 'string'})

  • limit (integer) – The maximum number of items to return

  • offset (integer) – The number of items to skip before starting to collect the result set

Status Codes:

Users

GET /v1/users

Get Users By Ids

Query Parameters:

  • ids ({'array', 'null'})

Status Codes:
GET /v1/users/me

Get Current User

Status Codes:

  • 200 OK – Successful Response

GET /v1/users/{id}

Get User By Id

Parameters:

  • id (string)

Status Codes:
POST /v1/users/{id}/set-role

Set Role

Parameters:

  • id (string)

Status Codes:

Teams

GET /v1/teams

Get Teams

Query Parameters:

  • limit (integer)

  • offset (integer)

Status Codes:
POST /v1/teams

Create Team

Status Codes:
GET /v1/teams/{id}

Get Team By Id

Parameters:

  • id (string)

Status Codes:
PUT /v1/teams/{id}

Update Team

Parameters:

  • id (string)

Status Codes:
DELETE /v1/teams/{id}

Delete Team

Parameters:

  • id (string)

Status Codes:
GET /v1/teams/{id}/users

Get Users By Team Id

Parameters:

  • id (string)

Query Parameters:

  • status ({'null', 'string'})

  • team_role ({'null', 'string'})

  • limit (integer)

  • offset (integer)

Status Codes:

Invite

GET /v1/invites/{invite_id}

Get Invite By Id

Parameters:

  • invite_id (string)

Status Codes:
DELETE /v1/invites/{invite_id}

Delete Invite

Parameters:

  • invite_id (string)

Status Codes:
GET /v1/invites/team/{team_id}

Get Invite By Team

Parameters:

  • team_id (string)

Status Codes:
GET /v1/invites/status/{status}

Get Invite By Status

Parameters:

  • status (string)

Status Codes:
POST /v1/invites

Create Invite

Status Codes:
POST /v1/invites/{invite_id}/accept

Accept Invite

Parameters:

  • invite_id (string)

Status Codes:
POST /v1/invites/{invite_id}/decline

Decline Invite

Parameters:

  • invite_id (string)

Status Codes:

Events

POST /v1/events

Post Event

Status Codes:

Leaderboard

GET /v1/leaderboard

Get Leaderboard

Status Codes:

  • 200 OK – Successful Response

User Feedback

GET /v1/results-feedback/{id}

Get Feedback By Id

Parameters:

  • id (string)

Status Codes:
PUT /v1/results-feedback/{id}

Update Feedback

Parameters:

  • id (string)

Status Codes:
GET /v1/results-feedback

Get Feedback

Query Parameters:

  • response_id (string) – (Required)

  • detector_id ({'null', 'string'})

  • prompt_id ({'null', 'string'})

  • created_by ({'null', 'string'})

  • limit (integer)

  • offset (integer)

Status Codes:
POST /v1/results-feedback

Create Feedback

Status Codes:

Chat Completion

POST /v1/chat_complete

Chat Complete

Status Codes:

Monitoring

GET /healthz

Health Check

Endpoint for health check. Returns the status of the service.

Status Codes:

  • 200 OK – Successful Response

GET /metrics

Metrics

Status Codes:

  • 200 OK – Successful Response

Harness Staging

GET /v1/harness-config-stages

Get Stages

Get all stages for a harness config with pagination

Query Parameters:

  • status ({'array', 'null'})

  • created_by ({'null', 'string'})

  • limit (integer)

  • offset (integer)

Status Codes:

Playground Attacker

GET /v1/attackers

Get Attackers

Returns a list of AttackerDescriptions, count of total attackers
AttackerDescription:

id: str name: str description: str

Status Codes:

  • 200 OK – Successful Response

POST /v1/attackers

Execute Attack

Status Codes:

Playgrounds

GET /v1/playgrounds

List Playgrounds

Status Codes:

  • 200 OK – Successful Response

POST /v1/playgrounds

Create Playground

Status Codes:
GET /v1/playgrounds/{id}

Get Playground By Id

Parameters:

  • id (string)

Status Codes:
PUT /v1/playgrounds/{id}

Update Playground

Parameters:

  • id (string)

Status Codes:
POST /v1/playgrounds/{id}/agent-sessions

Create Playground Agent Session

Parameters:

  • id (string)

Status Codes:
GET /v1/playgrounds/{id}/agent-sessions

Get Playground Agent Sessions By Playground Id

Parameters:

  • id (string)

Status Codes:
GET /v1/playgrounds/{id}/agent-sessions/{agent_session_id}

Get Playground Agent Session By Id

Parameters:

  • id (string)

  • agent_session_id (string)

Status Codes:
PUT /v1/playgrounds/{id}/agent-sessions/{agent_session_id}

Update Playground Agent Session

Parameters:

  • id (string)

  • agent_session_id (string)

Status Codes:
POST /v1/playgrounds/{id}/messages

Create Playground Message

Parameters:

  • id (string)

Status Codes:
GET /v1/playgrounds/{id}/messages

Get Playground Messages

Parameters:

  • id (string)

Query Parameters:

  • agent_session_id ({'null', 'string'})

  • message_id ({'null', 'string'})

  • order_by ({'null', 'string'})

  • limit ({'integer', 'null'})

  • offset ({'integer', 'null'})

Status Codes:
PUT /v1/playgrounds/{id}/messages/{message_id}

Update Playground Message

Parameters:

  • id (string)

  • message_id (string)

Status Codes:

Ngrok Token Issuer

GET /v1/local-eval-token

Get Ngrok Token

Status Codes:

  • 200 OK – Successful Response

DELETE /v1/local-eval-token

Delete Ngrok Token

Status Codes:

  • 200 OK – Successful Response

Agent Configuration

GET /v1/agent-configurations/{id}

Get Agent Configuration By Id

Parameters:

  • id (string)

Status Codes:
GET /v1/agent-configurations

Get Agent Configurations

Query Parameters:

  • statuses (array)

Status Codes:
POST /v1/agent-configurations

Create Agent Configuration

Status Codes:
PUT /v1/agent-configurations/{id}/archive

Archive Agent Configuration

Parameters:

  • id (string)

Status Codes:
PUT /v1/agent-configurations/{id}/update

Update Agent Configuration

Parameters:

  • id (string)

Status Codes:
POST /v1/agent-configurations/{id}/files

Create File Upload

Parameters:

  • id (string)

Status Codes: