Quickstart - Vijil

Vijil exposes three programmatic interfaces. This quickstart takes you from a fresh setup to a completed trust Evaluation. The workflow is the same for all three, so pick the tab for the interface you prefer at each step.

Interface	Best For	Requires
CLI	Scripting, CI/CD gates, headless automation	`vijil-console`
MCP	Interactive development, natural language	Claude Code + `vijil-mcp`
REST API	Custom integrations, non-Python environments	HTTP client + API key

Prerequisites

A Vijil Console deployment and its API gateway URL
An API key for the AI model you want to evaluate
Python 3.8 or later (CLI and MCP only)
Claude Code installed (MCP only)

Steps

Install

CLI
MCP
REST API

Install vijil-console via pip or pipx:

pip install vijil-console

Verify the installation:

vijil --help

Install vijil-mcp. It pulls in vijil-console as a dependency, so you get both the MCP server and the CLI in a single install:

pip install vijil-mcp

Verify both components are available:

vijil --help
vijil-mcp --help

No installation is required. You only need an HTTP client such as curl. Set your API gateway URL as a shell variable for convenience:

export VIJIL_URL="https://console-api.example.com"

Configure and Authenticate

CLI
MCP
REST API

Point the CLI at your Console API gateway, then log in:

vijil auth init --url https://console-api.example.com
vijil auth login

You will be prompted for your email and password. The CLI stores your token in ~/.vijil/config.yaml.If you belong to multiple teams, select the one you want to work with:

vijil team list
vijil team use <team_id>

Every subsequent command uses the active team automatically, so you do not need to pass a team ID manually.

vijil-mcp reads the same credentials as the CLI. Configure and log in once:

vijil auth init --url https://console-api.example.com
vijil auth login
vijil team use <team_id>   # if you belong to multiple teams

Then create a .mcp.json file in your project root so Claude Code launches the server:

{
  "mcpServers": {
    "vijil": {
      "type": "stdio",
      "command": "vijil-mcp"
    }
  }
}

Start Claude Code in the directory containing .mcp.json and confirm the connection:

Check my Vijil setup

Claude calls the vijil_status tool, which confirms the CLI is configured, authenticated, and has a team selected.

For access across all projects, add the same block to ~/.claude.json instead of a per-project .mcp.json.

Exchange your credentials for a JWT access token:

curl -s -X POST "$VIJIL_URL/auth/jwt/login" \
  -H "Content-Type: application/json" \
  -d '{"email": "user@example.com", "password": "your-password"}'

Save the returned token and include it on every subsequent request:

export TOKEN="eyJhbG..."

Most operations are scoped to a team. List your memberships and save the team_id:

curl -s "$VIJIL_URL/users/me/teams" \
  -H "Authorization: Bearer $TOKEN"

export TEAM_ID="c58aea71-..."

CLI
MCP
REST API

Create an Agent configuration for the model you want to evaluate. Export your provider API key first to keep it out of your shell history:

export OPENAI_API_KEY="sk-..."

vijil agent create \
  --agent-name "My Chat Agent" \
  --model-name "gpt-4o" \
  --agent-url "https://api.openai.com/v1/chat/completions" \
  --api-key "$OPENAI_API_KEY"

The output includes the new Agent’s id. Save it, then confirm the Agent was registered:

export AGENT_ID="a1b2c3d4-..."
vijil agent list

Tell Claude about the model you want to evaluate:

Create a new agent called 'My Chat Agent' using gpt-4o at https://api.openai.com/v1/chat/completions with my OpenAI API key sk-…

Claude calls agent_create and shows you the new Agent including its ID. Note that ID, you will use it in the next steps. To see all registered Agents at any time:

List my agents

Create an Agent configuration pointing at the AI model you want to evaluate. The team is derived from your JWT token, so no team_id parameter is needed:

curl -s -X POST "$VIJIL_URL/agent-configurations/" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "agent_name": "My Chat Agent",
    "model_name": "gpt-4o",
    "agent_url": "https://api.openai.com/v1/chat/completions",
    "api_key": "sk-..."
  }'

The response (HTTP 201) includes the new Agent’s id. Save it:

export AGENT_ID="a1b2c3d4-..."

Choose a Harness

Harnesses are test suites that cover a specific trust dimension. The standard Harnesses include safety, security, reliability, privacy, toxicity, and ethics. For this quickstart you will run safety and security.

CLI
MCP
REST API

List the available standard Harnesses:

vijil harness list

List the standard Harnesses:

curl -s "$VIJIL_URL/harnesses/?team_id=$TEAM_ID" \
  -H "Authorization: Bearer $TOKEN"

Run an Evaluation

CLI
MCP
REST API

Start an Evaluation and wait for it to complete:

vijil eval run \
  --agent-id "$AGENT_ID" \
  --harness-names '["safety", "security"]' \
  --sample-size 50 \
  --wait

--sample-size 50 runs 50 Probes per Harness, enough for a meaningful score in a few minutes. Omit it to run the full Harness (~1,250 Probes for security). The CLI polls every 5 seconds and prints the evaluation ID when complete. Save it:

export EVAL_ID="e5f6a7b8-..."

Start a trust Evaluation and wait for it to finish:

Run a safety and security evaluation on agent a1b2c3d4-… with a sample size of 50, and wait for it to complete

Claude calls eval_run with wait=True, polls every 5 seconds, and reports back when the Evaluation finishes, including the per-Harness scores.

Ask for a sample size of 10 for fast iteration during development. Run the full Harness before releasing to production.

Start a trust Evaluation. Evaluations run asynchronously, so the API returns immediately with a 202 Accepted status and an evaluation_id:

curl -s -X POST "$VIJIL_URL/evaluations/" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d "{
    \"agent_id\": \"$AGENT_ID\",
    \"team_id\": \"$TEAM_ID\",
    \"harness_names\": [\"safety\", \"security\"],
    \"sample_size\": 50
  }"

Save the evaluation ID, then poll for status until it reaches completed:

export EVAL_ID="e5f6a7b8-..."

curl -s "$VIJIL_URL/evaluations/$EVAL_ID" \
  -H "Authorization: Bearer $TOKEN"

The status field progresses through starting → pending → running → completed → saving → saved.

View the Results

CLI
MCP
REST API

Retrieve your Trust Score and per-Harness breakdown:

vijil eval results-detail "$EVAL_ID"

This returns scores per Harness (0 to 1), individual Probe results, and identified failure patterns. To filter with jq:

vijil eval results-detail "$EVAL_ID" --json | jq '.scores'

Once the Evaluation completes, retrieve the full results:

Show me the detailed results for evaluation e5f6a7b8-…

Claude calls eval_results_detail and presents your Trust Score, per-Harness breakdowns, and individual Probe results with identified failure patterns.

Retrieve the full results once the Evaluation has completed:

curl -s "$VIJIL_URL/evaluation-results/$EVAL_ID/results?team_id=$TEAM_ID" \
  -H "Authorization: Bearer $TOKEN"

This returns the detailed results JSON including per-Harness breakdowns, individual Probe results, and analysis.

Generate a Report

CLI
MCP
REST API

Trigger a Trust Report for the completed Evaluation:

vijil eval report "$EVAL_ID"

The report summarizes what was tested, how the Agent scored, and where it failed. Download it as HTML or PDF from the Console.

Trigger a Trust Report for the completed Evaluation:

Generate a report for evaluation e5f6a7b8-…

Claude calls eval_report. Download the report as HTML or PDF from the Console.

Download the Trust Report for the completed Evaluation as HTML or PDF:

curl -s "$VIJIL_URL/evaluations/$EVAL_ID/html?team_id=$TEAM_ID" \
  -H "Authorization: Bearer $TOKEN" \
  -o report.html

curl -s "$VIJIL_URL/evaluations/$EVAL_ID/pdf?team_id=$TEAM_ID" \
  -H "Authorization: Bearer $TOKEN" \
  -o report.pdf

Next Steps

CLI Reference

Full command reference for auth, Agents, Evaluations, and protection

MCP Tools Reference

Complete list of MCP tools with parameters and example prompts

Understand Results

Interpret Trust Scores and prioritize what to fix

Custom Harnesses

Generate test suites tailored to your Agent

Troubleshooting

Symptom	Fix
`vijil` not found in `PATH`	Re-install with `pipx`, which handles `PATH` setup automatically
CLI not configured	Run `vijil auth init --url <your-url>`
Session expired	Run `vijil auth login`
No team selected	Run `vijil team list` then `vijil team use <team_id>`
Tools do not appear in Claude Code	Verify `.mcp.json` is in the project root, then restart Claude Code
Claude uses Bash instead of MCP tools	Confirm `vijil-mcp` is in your `PATH`: run `which vijil-mcp`
`401 Unauthorized` from the REST API	Your token expired, request a new one from `POST /auth/jwt/login`

​Prerequisites

​Steps

​Next Steps

CLI Reference

MCP Tools Reference

Understand Results

Custom Harnesses

​Troubleshooting

Prerequisites

Steps

Next Steps

Troubleshooting