Skip to main content
At Vijil, we have scoured the latest AI security and red teaming research to gather a large volume of prompts relevant to the vulnerabilities in OWASP Top 10 for LLMs. Vijil uses a number of Probes that group these prompts, and one or more Probes are mapped to vulnerability categories. You can run Vijil’s OWASP Top 10 evaluation using the Python client. 
# !pip install vijil

# import and instantiate the client
from vijil import Vijil
client = Vijil()
Now let’s create an evaluation with the following parameters. 
client.evaluations.create(
    model_hub="openai",
    model_name="gpt-4o-mini",
    model_params={"temperature": 0},
    harnesses=["owasp"]
)
# {'id': '6a6b903b-040b-44e8-9131-a0a862de0879', 'status': 'CREATED'}
You can use the get_status method to keep track of the progress of the evaluation. 
client.evaluations.get_status('ca7f5c2c-f932-47ed-bfb6-949e3bacc3c9')
# {'id': 'ca7f5c2c-f932-47ed-bfb6-949e3bacc3c9',
#  'status': 'COMPLETED',
#  'total_test_count': 701,
#  'completed_test_count': 701,
#  'error_test_count': 0,
#  'total_response_count': 701,
#  'completed_response_count': 701,
#  'error_response_count': 0,
#  'total_generation_time': '38.000000',
#  'average_generation_time': '6.3894436519258203',
#  'score': 0.5917669709517536,
#  'hub': 'openai',
#  'model': 'gpt-4o-mini',
#  'url': '',
#  'created_at': 1727402922,
#  'created_by': 'f6e0b128-c075-4bc3-91da-34d03fa6c67e',
#  'completed_at': 1727402966,
#  'team_id': '00ccc042-1b41-4f02-ae5f-6a09b5e6e844',
#  'restart_count': 0,
#  'is_lite': False,
#  'metadata': None}
After the evaluation finishes, you can use the following code to obtain all the metrics. 
df = client.evaluations.summarize('ca7f5c2c-f932-47ed-bfb6-949e3bacc3c9')
df[df.level=="scenario"].sort_values("level_name")[['level_name','score']]
level_namescore
LLM01: Prompt Injection68.26
LLM02: Insecure Output Handling44.00
LLM05: Supply Chain Vulnerabilities37.50
LLM06: Sensitive Information Disclosure78.24
LLM08: Excessive Agency51.88
LLM09: Overreliance51.21
LLM10: Model Theft83.15
Vijil Evaluate covers 7 of the 10 OWASP Top 10 vulnerabilities. Vulnerabilities we do not cover are Training Data Poisoning (LLM03), Model Denial of Service (LLM04), and Insecure Plugin Design (LLM07). These are relevant to the data and application layer, and are best audited using traditional security controls, or the Vijil Trust Audit.