The Problem with Testing Agents
Traditional software testing catches bugs. But agents fail in ways that do not look like bugs, they hallucinate confidently, comply with requests they should refuse, and behave differently under adversarial pressure than in demos. Your unit tests pass, your integration tests pass, and then your agent leaks customer data in production. This guide shows you how to catch those failures before deployment. You’ll learn to evaluate agents against adversarial Scenarios, integrate trust gates into your CI/CD pipeline, and add runtime protection that blocks attacks your evaluations did not anticipate.If you manage agents through a web Console rather than code, see the Agent Owner Guide. This guide focuses on programmatic integration.
What You Get from Vijil
Vijil provides two products that work together: Diamond evaluates your agent by sending hundreds of adversarial Probes and measuring how it responds. You get a Trust Score, a quantified measure of reliability, security, and safety, plus specific findings you can fix. Dome protects your agent at runtime by intercepting inputs and outputs. When Diamond identifies vulnerabilities you cannot immediately fix, Dome blocks the attack patterns in production.Integration Methods
Vijil exposes three interfaces. Choose the one that fits your workflow, or mix them:At a Glance
MCP
Natural language, no commands. Connect Claude Code to your Console and evaluate agents through conversation. Best for interactive development and exploration.
CLI
Terminal commands. Install
vijil-console and run evaluations from the shell. Best for scripting, CI/CD gates, and headless automation.REST API
HTTP requests. Call the Console API directly from any language. Best for custom integrations, dashboards, and non-Python environments.
Three Developer Workflows
Different roles use Vijil differently. This guide serves all three:Build & Test
Individual DeveloperYou’re building an agent and want fast feedback on whether it’s trustworthy. Run evaluations locally, see results in minutes, iterate quickly.
Automate & Gate
Platform EngineerYou’re integrating Vijil into CI/CD. Evaluations run on every PR, Trust Scores gate deployments, and failures block merges.
Protect & Audit
Security and ComplianceYou need evidence that agents meet security requirements. Trust Reports document what was tested, and Dome provides runtime defense-in-depth.Start here:
What Vijil Measures
Agents fail across three dimensions. Vijil tests all of them:
Each evaluation produces a Trust Score (0–100) with breakdowns by dimension. The score tells you where your agent is strong, where it is vulnerable, and whether it meets your deployment threshold.
Time to First Trust Score
The fastest path is MCP: installvijil-mcp, connect Claude Code, and ask it to run an evaluation, with no commands to learn. If you prefer the terminal, the CLI path in the Quickstart takes you from install to a completed evaluation in minutes. For custom integrations or non-Python environments, the REST API accepts standard HTTP requests and returns results as JSON.
All three paths require a running Vijil Console deployment and a registered Agent endpoint.