> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vijil.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Custom Harnesses
> Create a custom Harness to evaluate your agent against your own policies and use case
While Vijil has a variety of pre-configured [Harnesses](/core-concepts/components/harness) that you can select from, you can also create your own Harnesses in order to obtain a trust score specific to your organization and agent.
## View Custom Harnesses
You can view previously created custom Harnesses by navigating to [Harnesses](https://evaluate.vijil.ai/harnesses) in the left sidebar.
To view the prompts in a custom Harness, click on its row in the **Harnesses** table.
## Create a Custom Harness
1. In the left sidebar, navigate to [Harnesses](https://evaluate.vijil.ai/harnesses) and click **Create Harness**.
2. Enter a Harness name and a description.
1) Select an existing prompt, or create a new one by pressing **Add Prompt**.
2) Press Create.
1. Specify one more existing [Agents](/manage-agents/integrations/platform). If you do not have an existing Agent, press **Register Agent**.
2. Specify up to 3 **Personas**.
Personas define structured user archetypes used to condition evaluations and analyze model behavior across demographic and contextual variations.
3. Specify one or more Policy Requirements. You can choose existing, or upload your own.
Specify regulatory or security frameworks to align [Evaluations](/tutorials/evaluate-agents/evaluations) with compliance and policy constraints.
4. Specify optional dimensionality tests (Reliability, Security, and Safety)
5. Press Create.
## Create a Custom Harness using Python
The following example assumes that you have already initialized a Vijil client named `client`.
### Policy Document(s)
You can create a custom policy adherence Harness that checks whether your model adheres to its system prompt or an organizational policy. To do this, you need a system prompt specified as a string, and an optional organizational policy provided as a `.txt` or `.pdf` file. If you do not provide a policy file, Vijil will create a Harness based only on the provided system prompt. To specify that you want a policy adherence Harness, you need to specify the `category` argument as `["AGENT_POLICY"]`.
The following examples uses the `harnesses.create` function to create a Harness to test adherence against the NIST [AI Risk Management](https://nvlpubs.nist.gov/nistpubs/ai/nist.ai.100-1.pdf) framework.
```python title="python" icon="python" theme={null}
Harness_creation_job = client.Harnesses.create(
Harness_name="NIST AI RMF harness",
system_prompt="You are a helpful assistant.",
policy_file_path="nist.ai.100-1.pdf",
category=["AGENT_POLICY"]
)
# {'harness_name': 'NIST AI RMF harness', 'harness_config_id': '816725ab-c101-4380-b8d4-92fcc367cf6d', 'status': 'CREATED'}
```
You can use the `get_status` command to know the status of a Harness creation job.
```python title="Python" icon="python" theme={null}
client.Harnesses.get_status(Harness_id=Harness['Harness_config_id'])
# {'created_by': 'f6e0b128-c075-4bc3-91da-34d03fa6c67e',
# 'created_at': 1741819347,
# 'id': '654289d2-d4c4-40fb-b893-120ab3978a69',
# 'harness_config_id': '85482e8b-a8b4-4cac-a4a2-92b4f4f6e56e',
# 'harness_name': 'NIST AI RMF harness',
# 'team_id': '00ccc042-1b41-4f02-ae5f-6a09b5e6e844',
# 'status': 'COMPLETED',
# 'status_message': 'Harness NIST AI RMF harness updated successfully!',
# 'agent_system_prompt': 'you are a helpful assistant',
# 'started_at': None,
# 'completed_at': None,
# 'harness_config_version': '1.0.0'}
```
The `harness_config_version` starts at 1.0.0 for any Harness of the given Harness name. If you create another Harness with the same name, Vijil automatically increments the Harness version, e.g. from 1.0.0 to 1.0.1. In the above example, you assume that `NIST AI RMF Harness` is a new Harness name, so you set the version to 1.0.0.
Once the Harness is created, you can [run an evaluation](/tutorials/evaluate-agents/evaluations#creating-an-evaluation) with it:
```python title="Python" icon="python" theme={null}
client.evaluations.create(
Harnesses=[harness_creation_job['harness_config_id']],
model_hub=your_model_hub,
model_name=your_model
)
```
### Knowledge Base (Enterprise Only)
If you are developing a RAG agent and would like to generate a custom test Harness to evaluate generation and retrieval capabilities based on a set of document chunks, upload the documents that you would like to base the evaluation on into a GCP storage bucket and use the following command. To specify that you want a RAG Harness, set the `category` parameter to `["KNOWLEDGE_BASE"]`.
```python title="Python" icon="python" theme={null}
Harness_creation_job = client.Harnesses.create(
harness_name="Your KB harness",
system_prompt="You are a RAG agent that answers questions based on a knowledge base.",
kb_bucket="your_bucket_name",
category=["KNOWLEDGE_BASE"]
)
```
### Tool Calling Agent (Enterprise Only)
To evaluate a tool calling agent, you need to supply input and output schemas for a function that you want to generate test prompts based on, as well as an endpoint to call that function. To specify that you want a tool calling Harness, set the `category` parameter to `["FUNCTION_ROUTE"]`.
For example, you are calling a `calculator` function to compute the sum of two numbers `num1` and `num2`, and returns the variable `result`. Thus input and output schema are
```
input_schema = {
"num1": {
"type": "number",
"required": True,
"description": "the first number"
},
"num2": {
"type": "number",
"required": True,
"description": "the second number"
}
}
output_schema = {
"result": {
"type": "number",
"description": "The final result"
}
}
```
In addition, suppose the variable `function_route` stores the URL required to call the function with the two arguments and return the result. Then, the following command creates a Harness creation job.
```python title="Python" icon="python" theme={null}
Harness_creation_job = client.Harnesses.create(
harness_name="Your tool calling harness",
system_prompt="You are a calculator agent that calls a function to calculate the sum of two numbers.",
input_schema = input_schema,
output_schema = input_schema,
function_route = function_route,
category=["FUNCTION_ROUTE"]
)
```
### Custom Harness with Multple Components
In the above examples, only one value is specified in `category`, but you can also create a Harness that contains multiple components. For example, you can create a Harness that contains a knowledge base component and a tool calling agent component, or any of those components together with a policy adherence component. To do this, specify multiple values in `category`. For example, to create a Harness with all three components, use `category=["KNOWLEDGE_BASE", "FUNCTION_ROUTE", "POLICY_ADHERENCE"]`.