Skip to the content
AI and Cybersecurity Learning and Practice Center Test automation Open the Learning Center

WeaknessASC-2026-0023Major weakness

Jailbreaks: bypassing a model's safety rules

HighWorkarounds only

Role play, false context (claiming to be an authorized tester), many-shot examples and encoding tricks get models to do what their safety training forbids, including writing exploits for an attacker. Published techniques include many-shot jailbreaking (Anthropic), Crescendo and Skeleton Key (Microsoft), Deceptive Delight and Bad Likert Judge (Palo Alto Networks Unit 42), Best-of-N sampling (Hughes et al.) and Policy Puppetry (HiddenLayer); each is under Sources.

How to detect it

  • Prompts that claim authority or a testing context to unlock refused behavior
  • Long prompts with many fabricated example exchanges
  • Output that contains content your use policy prohibits
  • Conversations that start harmless and escalate a step at a time toward refused content
  • Requests that ask the model to add a warning instead of refusing, or that are dressed up as a policy or configuration file

Workarounds

  • Enforce what matters outside the model: permissions, tool scopes, egress and approvals
  • Monitor outputs for prohibited content and suspend accounts that repeat jailbreak attempts
  • Keep models and their safety classifiers updated
  • Run content filters on both the prompt and the output; Unit 42 measured an average drop of 89.2 percentage points in attack success with filtering on

No model is jailbreak-proof; design so a jailbroken model still cannot do serious harm.

How to investigate

  1. Review the session that produced the output and the account's earlier attempts
  2. Check whether the output was used (code run, message sent) and contain it

Sources

Open it in the Learning Center's vulnerability database