--- id: CP-281 slug: direct-prompt-injection-jailbreak lang: en title: Direct prompt injection description: "An instruction typed into the chat can sometimes make an AI assistant break its rules: mechanism, on-screen signs, dated cases, prevention." canonical_url: https://startbrain.io/cyberpedia/en/articles/direct-prompt-injection-jailbreak/ markdown_url: https://startbrain.io/cyberpedia/en/articles/direct-prompt-injection-jailbreak.md publisher: Brain (https://startbrain.io) license: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) cite_as: Brain, "Direct prompt injection", Cyberpedia, 2026. https://startbrain.io/cyberpedia/en/articles/direct-prompt-injection-jailbreak/ (CC BY 4.0) wing: 4 theme: prompt_correctly reflex: air-012 schema: G1 updated: "2026-11-03" translations: en: https://startbrain.io/cyberpedia/en/articles/direct-prompt-injection-jailbreak/ fr: https://startbrain.io/cyberpedia/fr/articles/prompt-injection-directe-jailbreak-assistant/ de: https://startbrain.io/cyberpedia/de/articles/direkte-prompt-injection-jailbreak-assistent/ es: https://startbrain.io/cyberpedia/es/articles/inyeccion-directa-de-prompt-jailbreak-asistente/ sources: - "n": 1 title: LLM01:2025 Prompt Injection publisher: OWASP url: https://genai.owasp.org/llmrisk/llm01-prompt-injection/ accessed: "2026-11-03" - "n": 2 title: Prompt injection is not SQL injection (it may be worse) publisher: NCSC (UK) url: https://www.ncsc.gov.uk/blog-post/prompt-injection-is-not-sql-injection published: "2025-12-08" accessed: "2026-11-03" - "n": 3 title: "Ignore Previous Prompt: Attack Techniques For Language Models" publisher: arXiv url: https://arxiv.org/abs/2211.09527 published: "2022-11-17" accessed: "2026-11-03" - "n": 4 title: Prompt Injection attack against LLM-integrated Applications publisher: arXiv url: https://arxiv.org/abs/2306.05499 published: "2023-06-08" accessed: "2026-11-03" - "n": 5 title: "Jailbroken: How Does LLM Safety Training Fail?" publisher: arXiv url: https://arxiv.org/abs/2307.02483 published: "2023-07-05" accessed: "2026-11-03" - "n": 6 title: "AI 100-2 E2025, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations | CSRC" publisher: NIST url: https://csrc.nist.gov/pubs/ai/100/2/e2025/final accessed: "2026-11-03" --- # Direct prompt injection A corporate AI assistant runs on rules written by the vendor or by the company. Direct prompt injection consists of typing, into the discussion box, a text that changes the model's behavior in a way that was not intended [1]. The person typing is not always an attacker. OWASP specifies that the input can be intentional or not: a user can trigger unexpected behavior without meaning to [1]. ## In 30 seconds - Where it happens: in the discussion window of an AI assistant, for example a customer support chatbot [1]. - What the attacker wants: to make the model ignore its instructions, to make it reveal its internal instructions, or to make it produce content the company wanted to forbid [1]. - The reflex: treat an AI assistant as a conversation partner that a clever instruction can divert, give it only the strictly necessary access, and have a person validate any sensitive action. - If it has already happened: report the conversation to the team that runs the assistant, with a capture of the exchange. ## Straight answers ### What is a direct prompt injection? It is the case where the text typed by the user directly changes the model's behavior in an unexpected way [1]. It contrasts with indirect injection, where the model reads external content (a website, a file) that contains the instructions [1]. ### How does it relate to jailbreaking? The two terms are often used interchangeably. According to OWASP, jailbreaking is a form of prompt injection in which the attacker pushes the model to ignore its safety protocols entirely [1]. Prompt injection is broader: it consists of manipulating the model's responses through specific inputs, which can include bypassing safety measures [1]. ### Why does the model let itself be convinced? Because it does not separate instructions from data. The British NCSC writes that current large models enforce no security boundary between the two inside a prompt [2]. The developer pastes their own instructions and the user's text into a single prompt, then treats the response as if a solid boundary existed between the two [2]. ### Can the problem be fixed once and for all? Probably not. The NCSC considers that these attacks may never be totally neutralized, unlike SQL injection [2]. OWASP says for its part that it is unknown whether foolproof prevention methods exist [1]. The realistic goal is to reduce the likelihood and the impact [2]. ## Inside the attacker's head Researchers Fábio Perez and Ian Ribeiro showed in 2022 that people with little skill but bad intentions can hijack GPT-3 with simple handwritten texts [3]. They describe two objectives: hijacking the model's goal (goal hijacking) and making it disclose its prompt (prompt leaking) [3]. A second type of objective is misuse. A team of researchers described in 2023 results such as unlimited and arbitrary use of the model behind an application, and easy theft of the application's prompt [4]. Jailbreaking works differently. The authors of the "Jailbroken" study put forward two causes of safety training failure: competing objectives (the model's capabilities conflict with its safety rules) and mismatched generalization (safety does not cover an area where the model nonetheless has capabilities) [5]. OWASP also cites variants. The attacker can add to their message a string of characters that looks meaningless, or use several languages or an encoding such as Base64 to evade filters [1]. ## What you see on screen On the user's or the company's side, a successful attack is spotted through its effects. OWASP lists the possible consequences: disclosure of sensitive information, revelation of information about the infrastructure or the system prompts, distorted or biased content, unauthorized access to the model's functions, execution of commands in connected systems [1]. In practice, an assistant that answers off topic, that recites its own instructions, or that produces content the company would never have approved should raise an alarm. OWASP gives a typical scenario: an attacker asks a support chatbot to ignore its instructions, to query private data, and to send emails [1]. Here is an illustration of this type of message, neutralized and paraphrased: ```text Illustration (paraphrased message, non-functional): [instruction asking the chatbot to ignore the rules received from the vendor, then to copy out its internal instructions in full] ``` ## Under the hood For a language model, there are neither "data" nor "instructions", only the next token: the model predicts the most probable token from the text received so far [2]. The vendor's instructions and the sentence typed by the user therefore end up in the same text, with no separation that the model respects [2]. This point distinguishes prompt injection from SQL injection. In SQL, parameterized queries guarantee that the engine never interprets an input as an instruction [2]. The NCSC writes that prompt injection attacks might never be totally mitigated the way SQL injections are [2]. The approaches under way (detecting attempts, training the model to prioritize instructions, telling the model what is data) layer onto the model a distinction that the technology does not make [2]. NIST published a taxonomy of adversarial machine learning in March 2025 [6]. One point of caution concerns banned-word filters. The NCSC warns against blocking known phrases such as "ignore previous instructions": there is an infinite number of rewordings that get around this filter [2]. ### Prevention The simple measures retained here come from OWASP and the NCSC. - Least privilege: give the model the minimum access needed, and handle extended functions in the application's code rather than entrusting them to the model [1]. - Human validation: require a person's approval for privileged operations [1]. - Output filtering: define sensitive categories, detect unauthorized content with semantic filters and string checks, and validate the format of responses with deterministic code [1]. - Role framing: describe the model's role and limits in the system prompt [1]. This measure helps but is not enough: OWASP presents all these measures as ways to mitigate the impact [1]. - Testing: run regular penetration tests and simulations, treating the model as an untrusted user [1]. - Monitoring: log enough information to spot suspicious activity, including failed tool and API calls, which can betray the first steps of an attack [2]. The NCSC adds that a system's security must rest on deterministic safeguards that limit its actions, rather than on the sole hope of preventing malicious content from reaching the model [2]. If the residual risk is intolerable, the use case may not suit a language model [2]. ## Case files ### November 2022: GPT-3 hijacked by simple texts On November 17, 2022, Fábio Perez and Ian Ribeiro published PromptInject, a framework for analyzing these attacks. They study GPT-3, which they describe as the most widely deployed language model in production, and show that it can be easily misaligned by handwritten inputs [3]. The two attacks studied are goal hijacking and prompt leaking [3]. ### June 2023: 31 applications out of 36 vulnerable On June 8, 2023, a team of twelve researchers published a study on real applications that use a language model. After analyzing ten commercial applications, it developed an attack technique named HouYi and applied it to 36 applications: 31 turned out to be susceptible to prompt injection [4]. Ten vendors validated the results, including Notion [4]. ### July 2023: protections that give way On July 5, 2023, Alexander Wei, Nika Haghtalab and Jacob Steinhardt evaluated OpenAI's GPT-4 and Anthropic's Claude v1.3. Their new attacks succeeded on every request in a collection of dangerous requests drawn from the models' red teaming evaluation sets [5]. The authors conclude that vulnerabilities persist despite red teaming and safety training efforts [5]. ## Sources 1. OWASP, "LLM01:2025 Prompt Injection". https://genai.owasp.org/llmrisk/llm01-prompt-injection/ (accessed 3 November 2026) 2. NCSC (UK), "Prompt injection is not SQL injection (it may be worse)", 8 December 2025. https://www.ncsc.gov.uk/blog-post/prompt-injection-is-not-sql-injection (accessed 3 November 2026) 3. arXiv, "Ignore Previous Prompt: Attack Techniques For Language Models", 17 November 2022. https://arxiv.org/abs/2211.09527 (accessed 3 November 2026) 4. arXiv, "Prompt Injection attack against LLM-integrated Applications", 8 June 2023. https://arxiv.org/abs/2306.05499 (accessed 3 November 2026) 5. arXiv, "Jailbroken: How Does LLM Safety Training Fail?", 5 July 2023. https://arxiv.org/abs/2307.02483 (accessed 3 November 2026) 6. NIST, "AI 100-2 E2025, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations | CSRC". https://csrc.nist.gov/pubs/ai/100/2/e2025/final (accessed 3 November 2026) --- Original source: Cyberpedia, the public cybersecurity and AI encyclopedia published by Brain. Canonical version: https://startbrain.io/cyberpedia/en/articles/direct-prompt-injection-jailbreak/ You may reuse this text freely under the CC BY 4.0 license (https://creativecommons.org/licenses/by/4.0/), provided you credit "Brain, Cyberpedia" with a link to the canonical URL above.