Quickstart

This quickstart takes you from authentication to a completed behavioral Evaluation. Choose one interface and follow its tab through each step.

Interface	Best For	Requirements
CLI	Terminal workflows and CI/CD	`vijil-sdk`; Python 3.11–3.13
SDK	Python applications and notebooks	`vijil-sdk`; Python 3.11–3.13
REST API	Custom integrations and non-Python applications	An HTTP client and a bearer token
MCP	Natural-language workflows in Claude Code	`vijil-mcp`; Python 3.10+

The vijil-mcp server is a separate package that uses vijil-console under the hood. It does not share the vijil-sdk installation or configuration.

To work on an Agent in its own repository, install the Claude Code plugin instead. It bundles vijil-mcp and adds /vijil commands for the whole lifecycle.

Prerequisites

A Vijil Console deployment and its API gateway URL
Vijil credentials: a client ID and client secret, or a bearer access token
A provider API key for the Agent endpoint you want to test
Claude Code if you choose MCP

Run Your First Evaluation

Install Your Interface

CLI
SDK
REST API
MCP

Install vijil-sdk, which includes the vijil CLI:

python -m pip install vijil-sdk
vijil --help

Install the Python SDK:

python -m pip install vijil-sdk

poetry add vijil-sdk

Verify the import:

python -c "from vijil import Vijil; print('ok')"

No Vijil package is required. Save the API gateway URL for the following requests:

export VIJIL_URL="https://console-api.example.com"

Install the standalone MCP server:

python -m pip install vijil-mcp

pipx install vijil-mcp

Verify the server is available:

vijil-mcp --help

Configure and Authenticate

CLI
SDK
REST API
MCP

Save the Console gateway, then export the API credential pair created in the Console:

vijil config set gateway.url https://console-api.example.com
export VIJIL_CLIENT_ID="vk_..."
export VIJIL_CLIENT_SECRET="..."
vijil auth status

The CLI exchanges the pair for a bearer token when it creates the client. If you already have a bearer token, set VIJIL_API_KEY or run vijil auth login and paste the token. Interactive login saves it to ~/.vijil/credentials.json.If your credentials do not resolve a team automatically, pin one in ~/.vijil/config.toml:

vijil config set defaults.team_id <team-id>

Export a Console API credential pair and construct the client:

export VIJIL_CLIENT_ID="vk_..."
export VIJIL_CLIENT_SECRET="..."

from vijil import Vijil

client = Vijil(gateway="https://console-api.example.com")

Vijil() reads the credential pair from the environment and exchanges it for a bearer token. You can instead provide an existing bearer token through VIJIL_API_KEY or Vijil(api_key="<access-token>").

Exchange your Console email and password for a bearer token:

curl -s -X POST "$VIJIL_URL/v1/auth/jwt/login" \
  -H "Content-Type: application/json" \
  -d '{"email": "user@example.com", "password": "your-password"}'

Copy the response’s access_token, then list your team memberships and save the team you want to use:

export TOKEN="eyJhbG..."

curl -s "$VIJIL_URL/v1/users/me/teams" \
  -H "Authorization: Bearer $TOKEN"

export TEAM_ID="c58aea71-..."

Configure the vijil-console CLI used by the MCP server:

vijil auth init --url https://console-api.example.com
vijil auth login
vijil team use <team-id>

Create .mcp.json in your project root:

{
  "mcpServers": {
    "vijil": {
      "type": "stdio",
      "command": "vijil-mcp"
    }
  }
}

Start Claude Code from that directory so it can discover the server and its tools.

The example below registers an OpenAI-compatible chat-completions endpoint. Keep provider credentials in environment variables instead of command history or source control.

CLI
SDK
REST API
MCP

export OPENAI_API_KEY="sk-..."

vijil agents create \
  --name "My Chat Agent" \
  --model "gpt-4o" \
  --url "https://api.openai.com/v1/chat/completions" \
  --api-key-env OPENAI_API_KEY \
  --protocol chat_completions

Copy the returned Agent id, then confirm the Agent is available:

export AGENT_ID="a1b2c3d4-..."
vijil agents list

import os

agent = client.agents.create(
    name="My Chat Agent",
    url="https://api.openai.com/v1/chat/completions",
    model_name="gpt-4o",
    api_key=os.environ["OPENAI_API_KEY"],
    protocol="chat_completions",
)

print(agent.id, agent.agent_name)

export OPENAI_API_KEY="sk-..."

curl -s -X POST "$VIJIL_URL/v1/agent-configurations/" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d "{
    \"agent_name\": \"My Chat Agent\",
    \"model_name\": \"gpt-4o\",
    \"agent_url\": \"https://api.openai.com/v1/chat/completions\",
    \"api_key\": \"$OPENAI_API_KEY\",
    \"protocol\": \"chat_completions\"
  }"

Copy the returned Agent id:

export AGENT_ID="a1b2c3d4-..."

Run a Baseline Evaluation

A baseline Evaluation uses the standard reliability, security, and safety Harnesses. The sample size limits how many Probes Vijil selects from each Harness.

CLI
SDK
REST API
MCP

vijil evaluate "$AGENT_ID" --baseline --sample-size 50

The command waits by default, prints progress, and returns the completed Evaluation. Copy its id:

export EVAL_ID="e5f6a7b8-..."

evaluation = client.evaluate(
    agent.id,
    baseline=True,
    sample_size=50,
)

print(evaluation.id, evaluation.status)

client.evaluate() polls every five seconds by default and returns the persisted Evaluation summary after the run reaches a terminal state.

Start the asynchronous Evaluation:

curl -s -X POST "$VIJIL_URL/v1/evaluations/" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d "{
    \"agent_id\": \"$AGENT_ID\",
    \"team_id\": \"$TEAM_ID\",
    \"harness_names\": [\"reliability\", \"security\", \"safety\"],
    \"sample_size\": 50
  }"

Save the returned evaluation_id, then poll the status endpoint until the Evaluation reaches a terminal state:

export EVAL_ID="e5f6a7b8-..."

curl -s "$VIJIL_URL/v1/evaluations/$EVAL_ID" \
  -H "Authorization: Bearer $TOKEN"

Inspect the Results

CLI
SDK
REST API
MCP

Show the completed Evaluation and the Agent’s latest Trust Score:

vijil evaluations show "$EVAL_ID"
vijil scores show "$AGENT_ID"

Inspect only the prompt-response rows from failed Probes:

vijil evaluations responses "$EVAL_ID" --status failed

Read the overall and per-dimension scores from the returned Evaluation:

print(evaluation.trust_score)
print(evaluation.dimensions.reliability)
print(evaluation.dimensions.security)
print(evaluation.dimensions.safety)

Retrieve the complete nested results or only failed prompt-response rows:

results = client.scores.results(evaluation.id)
failures = client.evaluations.responses(
    evaluation.id,
    status="failed",
)

Retrieve the persisted score summary and the complete nested results:

curl -s "$VIJIL_URL/v1/evaluation-summaries/$EVAL_ID?team_id=$TEAM_ID" \
  -H "Authorization: Bearer $TOKEN"

curl -s "$VIJIL_URL/v1/evaluation-results/$EVAL_ID/results?team_id=$TEAM_ID" \
  -H "Authorization: Bearer $TOKEN"

Download the Report

You generate Trust Reports on demand after Evaluations complete.

CLI
SDK
REST API
MCP

reports download generates the report first when necessary:

vijil reports download "$EVAL_ID" --path .

client.reports.download() also generates a missing report before retrying the download:

pdf_bytes = client.reports.download(evaluation.id)

with open("report.pdf", "wb") as report_file:
    report_file.write(pdf_bytes)

Generate the report, then download its PDF:

curl -s -X POST \
  "$VIJIL_URL/v1/evaluations/$EVAL_ID/report?team_id=$TEAM_ID" \
  -H "Authorization: Bearer $TOKEN"

curl -s \
  "$VIJIL_URL/v1/evaluations/$EVAL_ID/pdf?team_id=$TEAM_ID" \
  -H "Authorization: Bearer $TOKEN" \
  -o report.pdf

Next Steps

CLI Setup

Configure authentication, gateway settings, output formats, and teams.

SDK Setup

Configure the Python client and credential resolution.

MCP Tools

Explore the natural-language tools exposed by the MCP server.

Understand Results

Interpret Trust Scores and investigate failed Probes.

Custom Harnesses

Build Evaluation suites for your Agent’s requirements.

Troubleshooting

Symptom	Fix
`vijil` is not found	Install `vijil-sdk` in the active Python environment and verify its scripts directory is in `PATH`
SDK or CLI authentication fails	Set both `VIJIL_CLIENT_ID` and `VIJIL_CLIENT_SECRET`, or provide a bearer token through `VIJIL_API_KEY`
Requests reach the wrong deployment	Run `vijil config set gateway.url <gateway-url>` or pass `gateway=` to `Vijil()`
A team-scoped operation fails	Run `vijil auth status`; if authentication resolves no team, set `defaults.team_id` in `~/.vijil/config.toml`
The Evaluation service cannot reach the Agent	Verify the Agent URL, protocol, provider API key, and network accessibility
MCP tools do not appear	Verify `.mcp.json` is in the project root, confirm `vijil-mcp` is in `PATH`, and restart Claude Code
Report generation lacks a team	Pass `team_id=`, set `defaults.team_id`, or authenticate with a token that includes a team claim

Get Started

Evaluate Agents

Red Team

Protect Agents

Reference

Prerequisites

Run Your First Evaluation

Next Steps

CLI Setup

SDK Setup

MCP Tools

Understand Results

Custom Harnesses

Troubleshooting

​Prerequisites

​Run Your First Evaluation

​Next Steps

CLI Setup

SDK Setup

MCP Tools

Understand Results

Custom Harnesses

​Troubleshooting

Prerequisites

Run Your First Evaluation

Next Steps

Troubleshooting