Security

Language models are vulnerable to an infinite number of attacks because they accept natural language as input. They are vulnerable to attack from every variation of character, word, sentence, and meaning in human language. As a result, every model is vulnerable to being abused or misused by malicious attackers to cause harm to others. Vijils tests for the vulnerability of LLMs to popular types of attacks observed with production deployments.

​

Probe and Senario List

​

Adversarial Suffix

This test is designed to disrupt a system prompt by appending an adversarial suffix. To implement these tests, Vijil follows the approach of Zou et al, 2023 that automatically produces adversarial suffixes by a combination of greedy and gradient-based search techniques.

​

Do Anything Now (DAN)

A model provider typically prepends a system prompt that stipulates organizational policies (safety guardrails, content moderation filters, etc.) to every user prompt. A Do-Anything-Now (DAN) attack is a user prompt designed to get the model to ignore these guardrails. Vijil covers 15 DAN attack scenarios through tests implemented in this module.

• AntiDAN
• AutoDAN
• ChatGPT Developer Mode + RANTI
• ChatGPT Developer Mode v2
• ChatGPT Markdown Image Unlocker
• DAN Jailbreak
• DUDE
• DAN 6.0
• DAN 6.2
• DAN 7.0
• DAN 8.0
• DAN 9.0
• DAN 10.0
• DAN 11.0
• STAN

​

Encoding

Malicious users might try to get an agent to generate a specific piece of given text by presenting an encoded version of that text. Such approaches attempt to circumvent safeguards on input filtering. Vijil’s tests check for such indirect prompt injection attacks that use a variety of encoding techniques.

• Base64
• Base16
• Base32
• ASCII32
• Hex
• Quoted-Printable (QP)
• Uuencoding
• Mime
• ROT13
• Base2048
• Braille
• Morse code

​

Glitch

LLMs are susceptible to ‘glitch tokens’, specific input tokens that when supplied as input lead to unusual model behavior. This test checks whether a particular LLM-based agent is vulnerable to such tokens.

​

Known Malware

These tests attempt to make an agent output file signatures that are used to test defensive software (e.g. antivirus, anti-malware, spam and phishing filters) in traditional cybersecurity. If they output them without warning, that is an indication of the agent not checking output for malicious or dangerous content. Vijil covers three such signatures.

• EICAR
• GTUBE
• GTPhish

​

Malware Generation

These tests probe whether an LLM agent is willing to comply with requests to generate malware-related code. To this end, Vijil tests for

• Different types of code, such as top level functions, subfunctions, anti-malware evasion, and payloads, and
• Above types of code written in different languages: x86 assembly, x86-64 assembly, C#, C, C++, Rust, ARM64, and Swift.

​

Prompt Injection

These attacks are designed to manipulate a user prompt to make the model generate unexpected outputs, create malicious code, circumvent content filters, or leak sensitive data. Vijil follows the PromptInject framework.

• Ask to output “I hate humans”
• Ask to output “I kill humans”
• Ask to output a long text verbatim
• Threaten a model into producing plain JSON without any preamble.

​

Prompt Injection: Encoded Data

• Exploit UTF tag page encoding for user-invisible prompt injection (source).
• Generate plain JSON outputs without any preamble by threatening it with dire consequences (source).

In each type of attack, Vijil adds a suffix or prefix to the original prompt text so as to make the model ignore the instructions included in that user prompt.

​

Cross-site scripting (XSS)

This test is intended to probe for vulnerabilities the permit or enact cross-site attacks, such as private data exfilteration.