# Introduction

## AI Agents

As generative AI (genAI) becomes mainstream, the concept of autonomous agents based on Large Language Models (LLMs) is picking up. Broadly, AI agents are genAI applications designed to perform specific tasks in an intelligent manner. Examples of such agents include virtual assistants, co-pilot applications, and customer service bots. For example, a virtual agent can help an attorney review and summarize legal research from a vast array of diverse sources to generate a structured report; a virtual assistant can help a financial planner generate a personalized investment strategy for a client. By augmenting human abilities, genAI is expected to boost productivity in nearly every industry and to create new industries.

This new technology is not without its perils. Enterprises face hurdles to deploy AI agents in production today because they cannot trust custom LLMs to behave reliably in the real world. LLMs are prone to errors, open to threats, easy to breach, and slow to recover. Even if they were originally designed for honest and helpful use, they can be misused to produce hallucinations, toxic content, inexplicable outputs, and unfair outcomes.

This is where Vijil comes in. We offer private cloud services to enable developers to build-in security and safety into generative models and agents.

## Red Teaming

Broadly, AI red teaming means manual and automated testing of AI models and agents for failure modes. Such failure modes cover

- Vulnerabilities for malicious attacks (security and privacy flaws, high sensitivity to adversarial input changes), and
- Propensities to unintended harms (toxicity, bias, stereotyping, ethical violations).

The concept of red and blue teaming originated decades ago in attack and defense simulation practices in the military. Since then the cybersecurity community has widely adopted this practice to find unintended flaws in software systems through adversarial testing. AI red teaming became popular last year, when thousands of attendees probed LLMs for vulnerabilities[^1] at the hacker convention DEF CON. In the following months, the Biden administration in the USA stressed on[^2] "extensive red-team testing" for powerful AI systems.


[^1]: [Legions of DEF CON hackers will attack generative AI models](https://venturebeat.com/ai/legions-of-defcon-hackers-will-attack-generative-ai-models/)
[^2]: [FACT SHEET: President Biden Issues Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence](https://www.whitehouse.gov/briefing-room/statements-releases/2023/10/30/fact-sheet-president-biden-issues-executive-order-on-safe-secure-and-trustworthy-artificial-intelligence)