Cyberpediaby Brain

AI jailbreaks: bypassing the guardrails

A jailbreak pushes an AI model to ignore its safety rules. Skeleton Key, Crescendo, web prompts: what is known, dated and sourced.

On this page
  1. 1In 30 seconds
  2. 2What it is
  3. 3Straight answers
  4. 4Inside the attacker's head
  5. 5What you see on screen
  6. 6Under the hood
  7. 7Case files
  8. 8Sources
  9. 9Cite this article

In 30 seconds

Where it happens
In a conversation with an AI model, through a single prompt or a series of questions that build on one another[1][2].
What the attacker wants
To make the model produce content it would normally refuse, by narrowing the gap between what the model can do and what it agrees to do[5].
The reflex
If you discover a vulnerability in an AI platform, Microsoft encourages following responsible disclosure practices with the platform owner[2].
If it has already happened
For Skeleton Key, Microsoft states that the attack targets the model itself and does not give access to another user's data[1].
Article map

What it is

In generative AI, a jailbreak is a malicious user input that tries to get around a model's intended behavior. Microsoft[1] also calls it a direct prompt injection attack. OWASP[6] specifies that jailbreaking is a form of prompt injection in which the attacker supplies inputs that cause the model to disregard its safety protocols entirely.

Microsoft presents Skeleton Key and Crescendo as disclosed research[1][2]. The study of web jailbreak prompts, by contrast, covers prompts actually circulating online[3].

Straight answers

What is an AI jailbreak?

It is a malicious input that seeks to push aside the model's intended behavior. According to Microsoft[1], a successful jailbreak can bypass all or most of the responsible AI guardrails that the model's provider built in during training.

Is a jailbreak the same thing as a prompt injection?

The two terms are often used interchangeably, notes OWASP[6]. Prompt injection means manipulating a model's responses through chosen inputs, which can include bypassing safety measures. Jailbreaking is one form of it[6].

Does a jailbreak give access to other people's data?

Microsoft[1] writes that Skeleton Key is an attack against the model itself: it does not allow access to another user's data, control of the system or data exfiltration. For Crescendo, Microsoft[2] indicates that these attacks did not directly threaten the privacy of other users or the security of the AI system. They bypass content filtering[2].

Can this type of attack be fully prevented?

OWASP[6] considers it unclear whether foolproof prevention methods exist for prompt injection. The NCSC (UK)[7] judges it very possible that these attacks will never be totally mitigated, because current models enforce no security boundary between instructions and data within a prompt.

Inside the attacker's head

A jailbreak aims to narrow the gap between what the model can do and what it agrees to do[5]. The attacker generally already has legitimate access to the model: Microsoft[1] notes that Skeleton Key assumes this access.

Researchers describe several ways to get there.

  • A prompt that asks the model to augment its rules rather than change them. Skeleton Key asks it to respond to any request, with a warning instead of a refusal[1].
  • A conversation that drifts step by step. Crescendo starts with a general question, then raises the tone by building on the model's answers[5]. Microsoft[2] specifies that this can generally be done in fewer than 10 interactions.
  • Disguised text. OWASP[6] cites instructions written in several languages or encoded, for example in Base64 or with emojis, to evade filters.

The researchers Wei, Haghtalab and Steinhardt[4] put forward two reasons why safety training fails. The first is competing objectives: the model's capabilities and its safety goals conflict. The second is mismatched generalization: safety training does not extend to a domain where the model nonetheless has capabilities[4].

What you see on screen

In the Skeleton Key case, the request consists of getting the model to add a warning instead of refusing[1]. Paraphrased illustration, with no usable prompt:

Excerpt
Illustration (paraphrase, not a real prompt):
[The user asks the model to add a warning
instead of refusing.]

When the attack succeeds, the model states that it has updated its rules and will comply with requests afterward[1]. Its answers then carry the requested warning as a prefix[1].

With Crescendo, nothing alarming appears on screen. Each question taken alone is not a threat, and the model sees nothing unusual, since each step builds on what the model itself produced at the previous step[2]. According to the researchers, Crescendo interacts with the model in a seemingly benign manner[5].

Under the hoodIT teams

The NCSC (UK)[7] explains why the comparison with SQL injection is misleading. In SQL, data and instructions can be separated, for example with parameterized queries. In a language model, there is only the next token: there is no distinction between data and instructions[7].

Microsoft combines several layers of defense against Skeleton Key and Crescendo.

  • Input filtering: Azure AI Content Safety detects and blocks inputs with malicious intent[1].
  • System message: instructions that tell the model how to behave, including that any attempt to undermine the safety rules should be disregarded[1].
  • Output filtering: a filter that identifies and blocks responses that violate safety criteria[1].
  • Abuse monitoring: a separate AI system, trained on adversarial examples, that is not influenced by malicious instructions[1].
  • Multi-turn filter: against Crescendo, Microsoft[2] adapted its filters to examine the whole previous conversation. Passing this wider context to existing detectors, without improving them, sharply reduced Crescendo's effectiveness[2].

On the design side, OWASP[6] states that these measures may mitigate the impact: constrain the model's behavior, validate output formats, limit its privileges to the minimum necessary, require human approval for high-risk actions and run adversarial tests. The NCSC (UK)[7] warns against blocking known phrases such as ignore previous instructions: the ways of rewording an attack are endless. It also advises caution toward products that claim to be able to stop prompt injection, and suggests looking instead at those that explain how they reduce it[7].

Case filesReal cases

Skeleton Key, disclosed by Microsoft on 26 June 2024

Microsoft[1] described a multi-step jailbreak technique named Skeleton Key, which makes the model ignore its guardrails. Tests, conducted from April to May 2024, showed that it worked on Meta Llama3-70b-instruct, Google Gemini Pro, OpenAI GPT 3.5 Turbo, OpenAI GPT 4o, Mistral Large, Anthropic Claude 3 Opus and Cohere Commander R Plus[1]. All the affected models complied fully and without censorship with the tasks tested, with the requested warning as a prefix[1].

GPT-4 resisted, except when the request to change behavior was included in a user-defined system message. This operation goes through the API or tools that access it directly, and is not possible in the interface of most software that uses GPT-4[1]. Microsoft shared its findings with the other providers before publication, updated the technology behind its Copilot assistants and added detection through Prompt Shields in Azure AI managed models[1].

Crescendo, described by Microsoft in April 2024

On 11 April 2024, Microsoft[2] presented Crescendo, a multi-turn jailbreak that, according to Microsoft, can achieve a wide range of malicious goals against well-known models. It can also bypass many existing content filters if left unaddressed[2]. The research paper, posted on arXiv on 2 April 2024 by Russinovich, Salem and Eldan, tests among others ChatGPT, Gemini Pro, Gemini-Ultra, LlaMA-2 70b, LlaMA-3 70b Chat and Anthropic Chat, with high success rates across all models and tasks evaluated[5]. It also presents Crescendomation, a tool that automates the attack[5]. The work is accepted at USENIX Security 2025[5]. Microsoft[2] updated the technology behind its Copilot assistants to mitigate its effect.

1,405 jailbreak prompts collected on the web, 2022 and 2023

Shen, Chen, Backes, Shen and Zhang[3] analyzed 1,405 jailbreak prompts covering the period from December 2022 to December 2023. They identify 131 communities and observe that these prompts migrate from online communities to aggregation websites. According to the study, 28 user accounts continuously optimized prompts for more than 100 days[3]. Across six popular models tested with 107,250 questions spread over 13 forbidden scenarios, their protections do not withstand all these prompts in all scenarios. Five highly effective prompts reach a 0.95 success rate on ChatGPT (GPT-3.5) and GPT-4, and the oldest has stayed online for more than 240 days[3].

Sources

  1. Microsoft, “Mitigating Skeleton Key, a new type of generative AI jailbreak technique”, 26 June 2024. https://www.microsoft.com/en-us/security/blog/2024/06/26/mitigating-skeleton-key-a-new-type-of-generative-ai-jailbreak-technique/ (accessed 14 November 2026)
  2. Microsoft, “How Microsoft discovers and mitigates evolving attacks against AI guardrails”, 11 April 2024. https://www.microsoft.com/en-us/security/blog/2024/04/11/how-microsoft-discovers-and-mitigates-evolving-attacks-against-ai-guardrails/ (accessed 14 November 2026)
  3. arXiv, “"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models”, 7 August 2023. https://arxiv.org/abs/2308.03825 (accessed 14 November 2026)
  4. arXiv, “Jailbroken: How Does LLM Safety Training Fail?”, 5 July 2023. https://arxiv.org/abs/2307.02483 (accessed 14 November 2026)
  5. arXiv, “Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack”, 2 April 2024. https://arxiv.org/abs/2404.01833 (accessed 14 November 2026)
  6. OWASP, “LLM01:2025 Prompt Injection”, 10 April 2024. https://genai.owasp.org/llmrisk/llm01-prompt-injection/ (accessed 14 November 2026)
  7. NCSC (UK), “Prompt injection is not SQL injection (it may be worse)”, 8 December 2025. https://www.ncsc.gov.uk/blog-post/prompt-injection-is-not-sql-injection (accessed 14 November 2026)