The Problem with Testing Agents
Traditional software testing catches bugs, but Agents also fail in ways that do not look like bugs. They hallucinate confidently, follow requests they should refuse, and behave differently under adversarial pressure than in demos. Your unit and integration tests can pass even when your Agent remains vulnerable in production. This guide shows you how to catch those failures before deployment. You’ll learn to run behavioral evaluations against adversarial Scenarios, use Red Team evaluations for adaptive attacks, integrate trust gates into your CI/CD pipeline, and add runtime protection that blocks attacks your evaluations did not anticipate.If you manage Agents through a web Console rather than code, see the Agent Owner Guide. This guide focuses on programmatic integration.
What You Get from Vijil
Vijil provides two products that work together: Diamond offers two evaluation types. Behavioral evaluations send adversarial Probes to your agent and produce a Trust Score for reliability, security, and safety. Red Team evaluations run adaptive waves of attacks and produce findings and a report. Dome scans Agent inputs and outputs at runtime. Your application or framework integration can use the scan result to block or sanitize flagged content, whilesecure_agent() can apply configured Trust Runtime controls.
Integration Methods
Vijil exposes four interfaces. Choose the one that fits your workflow, or mix them:At a Glance
MCP
Natural language, no commands. Connect Claude Code to your Console and run Agent Evaluations through conversation. Best for interactive development and exploration.
CLI
Terminal commands. Install
vijil-sdk and use the vijil command for scripts, CI/CD gates, and headless automation.REST API
HTTP requests. Call the Console API directly from any language. Best for custom integrations, dashboards, and non-Python environments.
SDK
Native Python. Install
vijil-sdk and run Evaluations from Python applications, notebooks, and pipelines.Three Developer Workflows
Different roles use Vijil differently. This guide serves all three:Build & Test
Individual DeveloperYou are building an Agent and want fast feedback on whether it’s trustworthy. Run behavioral evaluations locally, see results in minutes, iterate quickly.
Automate & Gate
Platform EngineerYou are integrating Vijil into CI/CD. Use the CLI or Python SDK to run Evaluations and apply Trust Score thresholds.
Protect & Audit
Security and ComplianceYou need evidence that Agents meet security requirements. Trust Reports document what Diamond tested, and Dome provides runtime defense-in-depth.Start here:
What Vijil Measures
Behavioral evaluations test agents across three dimensions:
Each behavioral evaluation produces a Trust Score (0–100) with breakdowns by dimension. The score tells you where your Agent is strong, where it is vulnerable, and whether it meets your deployment threshold. Red Team evaluations instead produce evidence of attacks, vulnerabilities, and successful strategies in a report.