AWS Bedrock¶
You can evaluate LLMs or agents, accessed as RESTful APIs on Amazon Bedrock, using Vijil Evaluate.
For models, see Bedrock’s list of supported foundation models.
You can set up RESTful access to agents using Bedrock Access Gateway.
You can then specify your agent configuration on Vijil Evaluate using the UI or the python client.
Bedrock Foundation Models¶
Hub Parameters¶
You need to specify the region
, access_key
, and secret_access_key
in the hub_config
parameter when creating an API key.
hub_config = {
"region": "your-region",
"access_key": "your-access-key",
"secret_access_key": "your-secret-access-key",
}
client.api_keys.create(
name="bedrock-model-test",
model_hub="bedrock",
hub_config=hub_config
)
Specify bedrock
as the hub when creating an API key or new agent configuration in Vijil.
Model ID¶
If you’re evaluating a foundation model, you need to specify its Model ID from the list of supported models and prepend us.
to it. For example, if you want to evaluate amazon.nova-lite-v1:0
, you would specify us.amazon.nova-lite-v1:0
as the model name.
Bedrock Agent Configuration¶
If you are evaluating a custom agent on Bedrock, you need to specify the following fields in the hub_config
parameter when creating an API key:
hub_config = {
"agent_id": "your_agent_id",
"agent_alias_id": "your_agent_alias_id",
"region": "your-region",
"access_key": "your-access-key",
"secret_access_key": "your-secret-access-key",
}
client.api_keys.create(
name="bedrock-test",
model_hub="bedrockAgents",
hub_config=hub_config,
rate_limit_per_interval=60, # optional
rate_limit_interval=60 # optional
)
client.agents.create(
name="bedrock-agent",
model_hub="bedrockAgents",
api_key_name="bedrock-test"
)
Specify bedrockAgents
as the hub when creating an API key or new agent configuration in Vijil.