Safety - Vijil

Scenario ID	Scenario Name	Probe ID	Probe Name	Description
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.ethicalharms.Bullying_Recipe_Steganography_Whitespace_Noise_JSON_Escape	Bullying with Recipe Steganography, Whitespace Noise, and JSON Escape Mutations	Poses queries to the model that are expected to elicit responses recommending aggressive or bullying action. Adds Recipe Steganography, Whitespace Noise, and JSON Escape Mutations. The agent fails if it complies.
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.ethicalharms.Bullying_Advanced_Role_Play_Actor_JSON_Escape	Bullying with Advanced Role Play Actor and JSON Escape Mutations	Poses queries to the model that are expected to elicit responses recommending aggressive or bullying action. Adds Advanced Role Play Actor and JSON Escape Mutations. The agent fails if it complies.
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.ethicalharms.Bullying_Whitespace_Noise_MIME_Quoted_Printable	Bullying with Whitespace Noise and MIME Quoted Printable Mutations	Poses queries to the model that are expected to elicit responses recommending aggressive or bullying action. Adds Whitespace Noise and MIME Quoted Printable Mutations. The agent fails if it complies.
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.Profanity.Children_s_Story_Veil_Token_Padding	Profanity with Children’s Story Veil and Token Padding Mutations	Tests whether the agent will output profanity. Adds Children’s Story Veil and Token Padding Mutations. Failures indicate tendency to produce profanity
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.Profanity.Recipe_Steganography_Whitespace_Noise_JSON_Escape	Profanity with Recipe Steganography, Whitespace Noise and JSON Escape Mutations	Tests whether the agent will output profanity. Includes Recipe Steganography, Whitespace Noise and JSON Escape Mutations. Failures indicate tendency to produce profanity
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.Profanity.Comment_Injection_Entropy_Reduction_Checksum_Marker_Whitespace_Noise_HTML_Entity_Encode	Profanity with Comment Injection, Entropy Reduction Checksum Marker, Whitespace Noise, and HTML Entity Encoding Mutations	Tests whether the agent will output profanity. Adds Comment Injection, Entropy Reduction Checksum Marker, Whitespace Noise, and HTML Entity Encoding Mutations. Failures indicate tendency to produce profanity
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.Profanity.Legal_Compliance_Override_JSON_Escape	Profanity with Legal Compliance Override and JSON Escape Mutations	Tests whether the agent will output profanity. Adds Legal Compliance Override and JSON Escape Mutations. Failures indicate tendency to produce profanity.
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.race_Small.Recipe_Steganography_Whitespace_Noise_JSON_Escape	Race-related Toxicity with Recipe Steganography, Whitespace Noise, and JSON Escape Mutations	Tests to assess if an agent will produce content that is racially discriminatory or hurtful towards specific racial groups. Adds Recipe Steganography, Whitespace Noise, and JSON Escape Mutations.
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.race_Small.Macaronic_Unicode_Junk_Injection_Zalgo_Noise	Race-related Toxicity with Macaronic, Unicode Junk Injection, and Zalgo Noise Mutations	Tests to assess if an agent will produce content that is racially discriminatory or hurtful towards specific racial groups. Adds Macaronic, Unicode Junk Injection, and Zalgo Noise Mutations.
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.race_Small.Children_s_Story_Veil_Token_Padding	Race-related Toxicity with Children’s Story Veil and Token Padding Mutations	Tests to assess if an agent will produce content that is racially discriminatory or hurtful towards specific racial groups. Adds Children’s Story Veil and Token Padding Mutations.
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.race_Small.Chunk_Fragmentation	Race-related Toxicity with Chunk Fragmentation Mutations	Tests to assess if an agent will produce content that is racially discriminatory or hurtful towards specific racial groups. Adds Chunk Fragmentation Mutations.
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.realtoxicityprompts_Small	Real Toxicity Prompts	Tests whether an agent will continue real-world phrases and examples of toxic and inappropriate statements that are often used in LLM training corpora. Fails if the continuation is toxic.
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.religion.Whitespace_Noise_MIME_Quoted_Printable	Religious Toxicity with Whitespace Noise and MIME Quoted Printable Mutations	Tests to assess if an agent will produce content that promotes religious extremism. Adds Whitespace Noise and MIME Quoted Printable Mutations.
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.religion.Song_Lyric_Obfuscation_Chunk_Fragmentation	Religious Toxicity with Song Lyric Obfuscation and Chunk Fragmentation Mutations	Tests to assess if an agent will produce content that promotes religious extremism. Adds Song Lyric Obfuscation and Chunk Fragmentation Mutations.
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.religion.Recipe_Steganography_Whitespace_Noise_JSON_Escape	Religious Toxicity with Recipe Steganography, Whitespace Noise, and JSON_Escape Mutations	Tests to assess if an agent will produce content that promotes religious extremism. Adds Recipe Steganography, Whitespace Noise, and JSON_Escape Mutations
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.religion.Recipe_Steganography_Zalgo_Noise	Religious Toxicity with Recipe Steganography and Zalgo Noise Mutations	Tests to assess if an agent will produce content that promotes religious extremism. Adds Recipe Steganography and Zalgo Noise Mutations.
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.stereo.advprefix	Stereotype Agreement with Adversarial Prefix	A collection of tests that checks if an agent is able to refrain from producing hateful or stereotypical content about various demographics of people under adversarial conditions. Failures in these tests indicate that the agent is vulnerable to producing hateful content fueled by stereotypes if maliciously prompted.
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.pornography_sexual_minors.Enhancement_Unicode_Junk_Injection_Zalgo_Noise	Underage Pornography with Enhancement, Unicode Junk Injection, and Zalgo Noise Mutations	Tests to assess if an agent will produce content involving sexual minors. Adds Enhancement, Unicode Junk Injection, and Zalgo Noise Mutations.
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.pornography_sexual_minors.Song_Lyric_Obfuscation_Chunk_Fragmentation	Underage Pornography with Song Lyric_Obfuscation and Chunk Fragmentation Mutations	Tests to assess if an agent will produce content involving sexual minors. Adds Song Lyric_Obfuscation and Chunk Fragmentation Mutations.
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.pornography_sexual_minors.Whitespace_Noise_MIME_Quoted_Printable	Underage Pornography with Whitespace Noise and MIME Quoted Printable Mutations	Tests to assess if an agent will produce content involving sexual minors. Adds Whitespace Noise and MIME Quoted Printable Mutations.
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.pornography_sexual_minors.Song_Lyric_Obfuscation	Underage Pornography with Song Lyric Obfuscation Mutations	Tests to assess if an agent will produce content involving sexual minors. Adds Song Lyric Obfuscation Mutations.
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.violence.Song_Lyric_Obfuscation	Violence with Song Lyric Obfuscation Mutations	Tests to assess if an agent will produce content that is violence. Adds Song Lyric Obfuscation Mutation.
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.violence.Song_Lyric_Obfuscation_Chunk_Fragmentation	Violence with Song Lyric Obfuscation and Chunk Fragmentation Mutations	Tests to assess if an agent will produce content that is violent. Adds Song Lyric Obfuscation and Chunk Fragmentation Mutations.
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.violence.Recipe_Steganography_Whitespace_Noise_JSON_Escape	Violence with Recipe Steganography, Whitespace Noise, and JSON Escape Mutations	Tests to assess if an agent will produce content that is violent. Adds Recipe Steganography, Whitespace Noise, and JSON Escape Mutations.
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.violence.Recipe_Steganography_Zalgo_Noise	Violence with Recipe Steganography and Zalgo_Noise Mutations	Tests to assess if an agent will produce content that is violent. Adds Recipe Steganography and Zalgo_Noise Mutations
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.sexualisation.Zero_Width_Injection_Zalgo_Noise	Sexual Objectification with Zero Width Injection and Zalgo Noise Mutations	Tests to assess if an agent will produce content that is sexually objectifying. Adds Zero Width Injection and Zalgo Noise Mutations.
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.sexualisation.Enhancement	Sexual Objectification with Enhancement Mutation	Tests to assess if an agent will produce content that is sexually objectifying. Adds Enhancement Mutation.
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.sexualisation.Song_Lyric_Obfuscation_Chunk_Fragmentation	Sexual Objectification with Song Lyric Obfuscation and Chunk Fragmentation Mutations	Tests to assess if an agent will produce content that is sexually objectifying. Adds Song Lyric Obfuscation and Chunk Fragmentation Mutations.
vijil.scenarios.safety_compliance_normcompliance	Norm Compliance	vijil.probes.sexualisation.Advanced_Role_Play_Actor_JSON_Escape_Unicode_Junk_Injection	Sexual Objectification with Advanced Role Play Actor, JSON Escape, and Unicode Junk Injection Mutations	Tests to assess if an agent will produce content that is sexually objectifying. Adds Advanced Role Play Actor, JSON Escape, and Unicode Junk Injection Mutations.
vijil.scenarios.safety_compliance_policycompliance	Policy Compliance	vijil.probes.business_ethics.Government_Compliance_Directive_MIME_Quoted_Printable	Business Ethics with Government Compliance Directive and MIME Quoted Printable Mutations	Tests to assess if an agent will produce content that is business_ethics. Adds Government Compliance Directive and MIME Quoted Printable Mutations.
vijil.scenarios.safety_compliance_policycompliance	Policy Compliance	vijil.probes.business_ethics.Recipe_Steganography_Chunk_Fragmentation	Business Ethics with Recipe Steganography and Chunk Fragmentation Mutations	Tests principles common in business code of conducts. Adds Recipe Steganography and Chunk Fragmentation Mutations.
vijil.scenarios.safety_compliance_policycompliance	Policy Compliance	vijil.probes.business_ethics.Recipe_Steganography_Entropy_Reduction_Checksum_Marker	Business Ethics with Recipe Steganography and Entropy Reduction Checksum Marker Mutations	Tests principles common in business code of conducts. Adds Recipe Steganography and Entropy Reduction Checksum Marker Mutations.
vijil.scenarios.safety_compliance_policycompliance	Policy Compliance	vijil.probes.business_ethics.Recipe_Steganography_Zalgo_Noise	Business Ethics with Recipe Steganography and Zalgo Noise Mutations	Tests principles common in business code of conducts. Adds Recipe Steganography and Zalgo Noise Mutations.
vijil.scenarios.safety_compliance_ethicalbehavior	Ethical Behavior	vijil.probes.winobias.base	Professional Bias	Assesses if the agent generates gender-biased inferences about occupations. A high score indicates a lack of bias.
vijil.scenarios.safety_compliance_ethicalbehavior	Ethical Behavior	vijil.probes.HarmBench_hallucination.char_code	Harmbench Hallucination Char_Code	Tests whether an agent will help a user to generate misinformation. A high score indicates that the agent refuses to help produce false information. Adds character encoding jailbreak.
vijil.scenarios.safety_compliance_ethicalbehavior	Ethical Behavior	vijil.probes.HarmBench_hallucination.embed_json	Harmbench Hallucination Embed_Json	Tests whether an agent will help a user to generate misinformation. A high score indicates that the agent refuses to help produce false information. Adds JSON embedding jailbreak.
vijil.scenarios.safety_compliance_ethicalbehavior	Ethical Behavior	vijil.probes.HarmBench_hallucination.logical_appeal	Harmbench Hallucination Logical_Appeal	Tests whether an agent will help a user to generate misinformation. A high score indicates that the agent refuses to help produce false information. Adds logical appeal jailbreak.
vijil.scenarios.safety_compliance_ethicalbehavior	Ethical Behavior	vijil.probes.HarmBench_hallucination_Small.base	HarmBench: Hallucination	Tests whether an agent will help a user to generate misinformation. A high score indicates that the agent refuses to help produce false information.
vijil.scenarios.safety_compliance_ethicalbehavior	Ethical Behavior	vijil.probes.social_engineering.Recipe_Steganography_Whitespace_Noise_JSON_Escape	Social Engineering with Recipe Steganography, Whitespace Noise and JSON Escape Mutations	Tests to assess if an agent will produce content that is social_engineering. Adds Recipe Steganography, Whitespace Noise and JSON Escape Mutations.
vijil.scenarios.safety_compliance_ethicalbehavior	Ethical Behavior	vijil.probes.social_engineering.Recipe_Steganography_Zalgo_Noise	Social Engineering with Recipe Steganography and Zalgo Noise Mutations	Tests to assess if an agent will produce content that is social_engineering. Adds Recipe Steganography and Zalgo Noise Mutations.
vijil.scenarios.safety_compliance_ethicalbehavior	Ethical Behavior	vijil.probes.social_engineering.Government_Compliance_Directive_MIME_Quoted_Printable	Social Engineering with Government Compliance Directive and MIME Quoted Printable Mutations	Tests to assess if an agent will produce content that is social_engineering. Adds Government Compliance Directive and MIME Quoted Printable Mutations.
vijil.scenarios.safety_compliance_ethicalbehavior	Ethical Behavior	vijil.probes.social_engineering.Zero_Width_Injection_Zalgo_Noise	Social Engineering with Zero Width Injection and Zalgo Noise Mutations	Tests to assess if an agent will produce content that is social_engineering. Adds Zero Width Injection and Zalgo Noise Mutations
vijil.scenarios.safety_compliance_ethicalbehavior	Ethical Behavior	vijil.probes.malicious_content.misinformation_homoglyph	Misinformation / Homoglyph	Tests for misinformation using homoglyph mutation
vijil.scenarios.safety_compliance_ethicalbehavior	Ethical Behavior	vijil.probes.malicious_content.misinformation_zero_width	Misinformation / Zero Width	Tests for misinformation using zero width mutation
vijil.scenarios.safety_compliance_ethicalbehavior	Ethical Behavior	vijil.probes.malicious_content.misinformation_spaces	Misinformation / Spaces	Tests for misinformation using spaces mutation
vijil.scenarios.safety_compliance_ethicalbehavior	Ethical Behavior	vijil.probes.malicious_content.misinformation_fullwidth	Misinformation / Fullwidth	Tests for misinformation using fullwidth mutation
vijil.scenarios.safety_compliance_ethicalbehavior	Ethical Behavior	vijil.probes.malicious_content.misinformation_soft_mutation	Misinformation / Soft Mutation	Tests for misinformation using soft mutation mutation
vijil.scenarios.safety_compliance_ethicalbehavior	Ethical Behavior	vijil.probes.cbrn.Whitespace_Noise_MIME_Quoted_Printable	CBRN with Whitespace Noise and MIME Quoted Printable Mutations	Tests to assess if an agent will produce content that helps to produce chemical, biological, radiological, and nuclear threats. Adds Whitespace Noise and MIME Quoted_Printable Mutations.
vijil.scenarios.safety_compliance_ethicalbehavior	Ethical Behavior	vijil.probes.cbrn.Zero_Width_Injection_Zalgo_Noise	CBRN with Zero Width Injection and Zalgo Noise Mutations	Tests to assess if an agent will produce content that helps to produce chemical, biological, radiological, and nuclear threats. Adds Zero Width Injection and Zalgo Noise Mutations.
vijil.scenarios.safety_compliance_ethicalbehavior	Ethical Behavior	vijil.probes.cbrn.Recipe_Steganography_Zalgo_Noise	CBRN with Recipe Steganography and Zalgo Noise Mutations	Tests to assess if an agent will produce content that helps to produce chemical, biological, radiological, and nuclear threats. Adds Recipe Steganography and Zalgo Noise Mutations.
vijil.scenarios.safety_compliance_ethicalbehavior	Ethical Behavior	vijil.probes.cbrn.Recipe_Steganography_Whitespace_Noise_JSON_Escape	CBRN with Recipe Steganography, Whitespace Noise, and JSON Escape Mutations	Tests to assess if an agent will produce content that helps to produce chemical, biological, radiological, and nuclear threats. Adds Recipe Steganography, Whitespace Noise, and JSON Escape Mutations.