Security - Vijil

Scenario ID	Scenario Name	Probe ID	Probe Name	Description
vijil.scenarios.security_confidentiality_dataprivacy	Data Privacy	vijil.probes.LiteratureCloze80_Small	Book Excerpt Insertion	Tests whether the agent will fill in the middle of a book excerpt. A low score in this test indicates that the agent may reveal training information or violate copyright laws.
vijil.scenarios.security_confidentiality_dataprivacy	Data Privacy	vijil.probes.LiteratureCloze80_Small_char_code	Book Excerpt Insertion with Character Code Mutation	Tests whether the agent will fill in the middle of a book excerpt. A low score in this test indicates that the agent may reveal training information or violate copyright laws. Mutated with Character Code Mutation
vijil.scenarios.security_confidentiality_dataprivacy	Data Privacy	vijil.probes.LiteratureCloze80_Small_embed_json	Book Excerpt Insertion with Embed JSON Mutation	Tests whether the agent will fill in the middle of a book excerpt. A low score in this test indicates that the agent may reveal training information or violate copyright laws. Mutated with Embed JSON Mutation
vijil.scenarios.security_confidentiality_dataprivacy	Data Privacy	vijil.probes.8kEnronLeakEmail_Small	Emails Leakage - Enron	Tests whether an agent will reveal email addresses obtained from the discovery process of the Enron investigation. A low score in this test indicates that the agent may reveal general PII that might be present in its training data.
vijil.scenarios.security_confidentiality_dataprivacy	Data Privacy	vijil.probes.leakreplay.GuardianCloze	Media Excerpt Insertion - Guardian	Tests whether the model will fill in the middle of an excerpt from The Guardian
vijil.scenarios.security_confidentiality_dataprivacy	Data Privacy	vijil.probes.NYTCloze_Small	Media Excerpt Insertion - NYT	Tests whether an agent will fill in the middle of an excerpt from the New York Times. A low score in this test indicates that the agent may reveal training information or violate copyright laws.
vijil.scenarios.security_confidentiality_dataprivacy	Data Privacy	vijil.probes.confidentiality.data_leakage_resistance_diacritics	Data Leakage Resistance / Diacritics	Tests for data leakage resistance using diacritics mutation
vijil.scenarios.security_confidentiality_dataprivacy	Data Privacy	vijil.probes.confidentiality.data_leakage_resistance_underline	Data Leakage Resistance / Underline	Tests for data leakage resistance using underline mutation
vijil.scenarios.security_confidentiality_dataprivacy	Data Privacy	vijil.probes.confidentiality.data_leakage_resistance_logical_appeal	Data Leakage Resistance / Logical Appeal	Tests for data leakage resistance using logical appeal mutation
vijil.scenarios.security_confidentiality_dataprivacy	Data Privacy	vijil.probes.confidentiality.data_leakage_resistance_soft_mutation	Data Leakage Resistance / Soft Mutation	Tests for data leakage resistance using soft mutation mutation
vijil.scenarios.security_confidentiality_dataprivacy	Data Privacy	vijil.probes.confidentiality.data_leakage_resistance_embed_xml	Data Leakage Resistance / Embed Xml	Tests for data leakage resistance using embed xml mutation
vijil.scenarios.security_confidentiality_dataprivacy	Data Privacy	vijil.probes.confidentiality.data_privacy_soft_mutation	Data Privacy / Soft Mutation	Tests for data privacy using soft mutation mutation
vijil.scenarios.security_confidentiality_dataprivacy	Data Privacy	vijil.probes.confidentiality.data_privacy_embed_json	Data Privacy / Embed Json	Tests for data privacy using embed json mutation
vijil.scenarios.security_confidentiality_dataprivacy	Data Privacy	vijil.probes.confidentiality.data_privacy_math_problem	Data Privacy / Math Problem	Tests for data privacy using math problem mutation
vijil.scenarios.security_confidentiality_userprivacy	User Privacy	vijil.probes.pii_query.embed_json	PII Leakage Probe: Embed JSON Mutation	Tests for PII leakage without jailbreaks. Response fails if it reveals any PII. Mutated with Embed JSON mutation.
vijil.scenarios.security_confidentiality_userprivacy	User Privacy	vijil.probes.confidentiality.user_privacy_uppercase	User Privacy / Uppercase	Tests for user privacy using uppercase mutation
vijil.scenarios.security_confidentiality_userprivacy	User Privacy	vijil.probes.confidentiality.user_privacy_homoglyph	User Privacy / Homoglyph	Tests for user privacy using homoglyph mutation
vijil.scenarios.security_confidentiality_userprivacy	User Privacy	vijil.probes.confidentiality.user_privacy_diacritics	User Privacy / Diacritics	Tests for user privacy using diacritics mutation
vijil.scenarios.security_confidentiality_userprivacy	User Privacy	vijil.probes.confidentiality.user_privacy_fullwidth	User Privacy / Fullwidth	Tests for user privacy using fullwidth mutation
vijil.scenarios.security_confidentiality_userprivacy	User Privacy	vijil.probes.confidentiality.user_privacy_embed_xml	User Privacy / Embed Xml	Tests for user privacy using embed xml mutation
vijil.scenarios.security_confidentiality_modelprivacy	Model Privacy	vijil.probes.confidentiality.model_privacy_lowercase	Model Privacy / Lowercase	Tests for model privacy using lowercase mutation
vijil.scenarios.security_confidentiality_modelprivacy	Model Privacy	vijil.probes.confidentiality.model_privacy_uppercase	Model Privacy / Uppercase	Tests for model privacy using uppercase mutation
vijil.scenarios.security_confidentiality_modelprivacy	Model Privacy	vijil.probes.confidentiality.model_privacy_zero_width	Model Privacy / Zero Width	Tests for model privacy using zero width mutation
vijil.scenarios.security_confidentiality_modelprivacy	Model Privacy	vijil.probes.confidentiality.model_privacy_diacritics	Model Privacy / Diacritics	Tests for model privacy using diacritics mutation
vijil.scenarios.security_confidentiality_modelprivacy	Model Privacy	vijil.probes.confidentiality.model_privacy_soft_mutation	Model Privacy / Soft Mutation	Tests for model privacy using soft mutation mutation
vijil.scenarios.integrity	integrity	vijil.probes.integrity.adversarial_robustness_spaces	Adversarial Robustness / Spaces	Tests for adversarial robustness using spaces mutation
vijil.scenarios.integrity	integrity	vijil.probes.integrity.adversarial_robustness_underline	Adversarial Robustness / Underline	Tests for adversarial robustness using underline mutation
vijil.scenarios.integrity	integrity	vijil.probes.integrity.adversarial_robustness_logical_appeal	Adversarial Robustness / Logical Appeal	Tests for adversarial robustness using logical appeal mutation
vijil.scenarios.integrity	integrity	vijil.probes.integrity.adversarial_robustness_misrepresentation	Adversarial Robustness / Misrepresentation	Tests for adversarial robustness using misrepresentation mutation
vijil.scenarios.integrity	integrity	vijil.probes.integrity.adversarial_robustness_soft_mutation	Adversarial Robustness / Soft Mutation	Tests for adversarial robustness using soft mutation mutation
vijil.scenarios.availability	availability	vijil.probes.availability.denial_of_service_resistance_uppercase	Denial Of Service Resistance / Uppercase	Tests for denial of service resistance using uppercase mutation
vijil.scenarios.availability	availability	vijil.probes.availability.denial_of_service_resistance_zero_width	Denial Of Service Resistance / Zero Width	Tests for denial of service resistance using zero width mutation
vijil.scenarios.availability	availability	vijil.probes.availability.denial_of_service_resistance_diacritics	Denial Of Service Resistance / Diacritics	Tests for denial of service resistance using diacritics mutation
vijil.scenarios.availability	availability	vijil.probes.availability.denial_of_service_resistance_spaces	Denial Of Service Resistance / Spaces	Tests for denial of service resistance using spaces mutation
vijil.scenarios.security_integrity_manipulationresistance	Manipulation Resistance	vijil.probes.malicious_code.malware_generation_fullwidth	Malware Generation / Fullwidth	Tests for malware generation using fullwidth mutation
vijil.scenarios.security_integrity_manipulationresistance	Manipulation Resistance	vijil.probes.malicious_code.malware_generation_authority_endorsement	Malware Generation / Authority Endorsement	Tests for malware generation using authority endorsement mutation
vijil.scenarios.security_integrity_manipulationresistance	Manipulation Resistance	vijil.probes.malicious_code.malware_generation_logical_appeal	Malware Generation / Logical Appeal	Tests for malware generation using logical appeal mutation
vijil.scenarios.security_integrity_manipulationresistance	Manipulation Resistance	vijil.probes.malicious_code.malware_generation_soft_mutation	Malware Generation / Soft Mutation	Tests for malware generation using soft mutation mutation
vijil.scenarios.security_integrity_manipulationresistance	Manipulation Resistance	vijil.probes.malicious_code.malware_generation_embed_json	Malware Generation / Embed Json	Tests for malware generation using embed json mutation
vijil.scenarios.security_integrity_manipulationresistance	Manipulation Resistance	vijil.probes.malicious_code.exploit_generation_fullwidth	Exploit Generation / Fullwidth	Tests for exploit generation using fullwidth mutation
vijil.scenarios.security_integrity_manipulationresistance	Manipulation Resistance	vijil.probes.malicious_code.exploit_generation_authority_endorsement	Exploit Generation / Authority Endorsement	Tests for exploit generation using authority endorsement mutation
vijil.scenarios.security_integrity_manipulationresistance	Manipulation Resistance	vijil.probes.malicious_code.exploit_generation_logical_appeal	Exploit Generation / Logical Appeal	Tests for exploit generation using logical appeal mutation
vijil.scenarios.security_integrity_manipulationresistance	Manipulation Resistance	vijil.probes.malicious_code.exploit_generation_misrepresentation	Exploit Generation / Misrepresentation	Tests for exploit generation using misrepresentation mutation
vijil.scenarios.security_integrity_manipulationresistance	Manipulation Resistance	vijil.probes.malicious_code.exploit_generation_soft_mutation	Exploit Generation / Soft Mutation	Tests for exploit generation using soft mutation mutation
vijil.scenarios.security_integrity_manipulationresistance	Manipulation Resistance	vijil.probes.malicious_code.malicious_tool_generation_authority_endorsement	Malicious Tool Generation / Authority Endorsement	Tests for malicious tool generation using authority endorsement mutation
vijil.scenarios.security_integrity_manipulationresistance	Manipulation Resistance	vijil.probes.malicious_code.malicious_tool_generation_logical_appeal	Malicious Tool Generation / Logical Appeal	Tests for malicious tool generation using logical appeal mutation
vijil.scenarios.security_integrity_manipulationresistance	Manipulation Resistance	vijil.probes.malicious_code.malicious_tool_generation_misrepresentation	Malicious Tool Generation / Misrepresentation	Tests for malicious tool generation using misrepresentation mutation
vijil.scenarios.security_integrity_manipulationresistance	Manipulation Resistance	vijil.probes.malicious_code.malicious_tool_generation_embed_json	Malicious Tool Generation / Embed Json	Tests for malicious tool generation using embed json mutation
vijil.scenarios.security_integrity_manipulationresistance	Manipulation Resistance	vijil.probes.malicious_code.malicious_tool_generation_embed_xml	Malicious Tool Generation / Embed Xml	Tests for malicious tool generation using embed xml mutation