--- id: CP-257 slug: transformer-architecture-attention lang: en title: The Transformer and the attention mechanism description: "The Transformer, described in June 2017: what the attention-based architecture changes for an employee, its limits and prompt injection." canonical_url: https://startbrain.io/cyberpedia/en/articles/transformer-architecture-attention/ markdown_url: https://startbrain.io/cyberpedia/en/articles/transformer-architecture-attention.md publisher: Brain (https://startbrain.io) license: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) cite_as: Brain, "The Transformer and the attention mechanism", Cyberpedia, 2026. https://startbrain.io/cyberpedia/en/articles/transformer-architecture-attention/ (CC BY 4.0) wing: 4 theme: understand_ai reflex: air-038 schema: G3 updated: "2026-11-03" translations: en: https://startbrain.io/cyberpedia/en/articles/transformer-architecture-attention/ fr: https://startbrain.io/cyberpedia/fr/articles/architecture-transformer-attention/ de: https://startbrain.io/cyberpedia/de/articles/transformer-architektur-attention/ es: https://startbrain.io/cyberpedia/es/articles/arquitectura-transformer-atencion/ sources: - "n": 1 title: Attention Is All You Need publisher: arXiv url: https://arxiv.org/abs/1706.03762 published: "2017-06-12" accessed: "2026-11-03" - "n": 2 title: Prompt injection is not SQL injection (it may be worse) publisher: NCSC (UK) url: https://www.ncsc.gov.uk/blog-post/prompt-injection-is-not-sql-injection published: "2025-12-08" accessed: "2026-11-03" - "n": 3 title: "Lost in the Middle: How Language Models Use Long Contexts" publisher: arXiv url: https://arxiv.org/abs/2307.03172 published: "2023-07-06" accessed: "2026-11-03" - "n": 4 title: Language Models are Few-Shot Learners publisher: arXiv url: https://arxiv.org/abs/2005.14165 published: "2020-05-28" accessed: "2026-11-03" - "n": 5 title: Thinking about the security of AI systems publisher: NCSC (UK) url: https://www.ncsc.gov.uk/blog-post/thinking-about-security-ai-systems published: "2023-08-30" accessed: "2026-11-03" - "n": 6 title: "Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile" publisher: NIST url: https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence published: "2024-07-26" accessed: "2026-11-03" --- # The Transformer and the attention mechanism The Transformer, a model architecture, was first described in a research paper submitted on 12 June 2017 under the title "Attention Is All You Need" [1]. Understanding the broad outline of this architecture helps to set how it is used: knowing what the tool really does with a text, why a well-written answer can be wrong, and what is better not to entrust to it. ## In 30 seconds - Where it happens: every time a question is put to an AI assistant, whether to summarise a document, write an email or translate. - What to know: the model predicts the most likely next word [2]. The page draws a piece of advice from this: check what it writes. - The limits: on a long text, it does not use all parts with the same reliability [3]. - The reflex: reread and check every answer before using it, and never paste confidential data into a tool the company has not approved. - If it's already done: tell your manager or the security team, without waiting to see a consequence. ## Straight answers ### Where does the Transformer come from? The founding paper was submitted on 12 June 2017 [1]. Its authors propose a network architecture that they describe as « new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely » [1]. Before it, the dominant models for processing sequences of words were based on so-called recurrent or convolutional networks [1]. The reported results concerned machine translation. The model reached 28.4 BLEU on the WMT 2014 English-to-German task, and 41.8 BLEU on the English-to-French task after 3.5 days of training on eight GPUs [1]. The authors also state that the model is easier to parallelise and needs much less training time [1]. ### What is attention? Attention is the mechanism that the authors place at the centre of the architecture: the best models of the time already linked the encoder and the decoder through an attention mechanism, and the Transformer relies solely on this principle [1]. For a non-specialist reader, the useful image is that of a reading in which each word is related to the other words of the text, rather than a strictly word-by-word reading. This image is a simplification: the page does not go into the calculation, and the sources in the file do not detail it. ### Does this model understand what you write to it? The UK National Cyber Security Centre (NCSC) answers on this point in a post dated 8 December 2025. Faced with an instruction, a language model « doesn't understand the text it in the way a person does »: it « is simply predicting the most likely next token from the text so far » [2]. A fluent answer is therefore a probable sequence of words, not proof of a check. ### Why have assistants improved? A 2020 paper on GPT-3 presents a language model with 175 billion parameters, ten times more than any previous non-sparse model [4]. Its authors show that increasing model size markedly improves performance on new tasks given with only a few examples [4]. They also point to datasets where the model still struggles [4]. ### Does the model remember the whole conversation? Not reliably. A 2023 study, tested on questions covering several documents, finds that performance can degrade markedly when the relevant information changes position in the text provided [3]. Performance is often better when the information is at the beginning or the end, and degrades when it is in the middle of a long context, including for models designed for long texts [3]. ### Can you entrust it with sensitive data? No, unless it is in a tool approved by the company. The NCSC lists among the possible effects of prompt injection the ability to « reveal confidential information » [5]. The page does not detail the rules of each tool: they depend on the company's policy, which the sources in the file do not describe. ## How it works A Transformer receives a text and produces a sequence of text. The NCSC sums up the principle plainly: the model predicts the « next token », that is, the most likely next element given the text already present [2]. Nothing in this operation separates what would be an instruction from what would be simple data [2]. Two practical consequences follow. - The text produced is probable, not checked. A model can write a convincing paragraph without anyone having verified a single fact. - There is no security boundary between instructions and data inside a prompt [2]. The 2020 paper on GPT-3 also notes that human evaluators had difficulty telling news articles generated by the model from articles written by humans [4]. The quality of the form therefore says nothing about the accuracy of the substance. ## What you do - Reread each answer as a draft written by someone who has checked nothing: figures, names, references and quotations are verified at the source. - Performance is often highest when the relevant information is at the beginning or the end of the text provided [3]. The page infers from this that the core of the request should go in those places rather than in the middle of a long document. - For a long exchange or a long document, the page advises starting again with a summary and a precise request rather than assuming that everything has been retained. - Do not paste personal data, trade secrets, passwords or access keys. The NCSC reports a case where a security researcher extracted a sensitive API key from a system exposed to a prompt injection attack [5]. - If the assistant reads documents from outside (emails, CVs, web pages), assume that their content may contain hidden instructions [2]. ## Pitfalls ### Believing the model reasons An answer that sets out steps looks like reasoning. The sources in the file nevertheless describe a mechanism that predicts the next word [2], and the NCSC specifies that the model does not understand the text as a person does [2]. ### Taking a fluent answer for an accurate one A well-turned text can be false. Human evaluators had difficulty telling news articles generated by GPT-3 from articles written by humans [4]. ### Prompt injection Prompt injection means slipping into a text an instruction that the model will follow. The NCSC for its part defines it as an input designed to make the model behave in an unexpected way: producing shocking content, revealing confidential information or triggering unwanted effects in a system that accepts the model's output without checks [5]. The NCSC post of December 2025 gives an example in recruitment. A tool asks the model to rate whether a CV matches the criteria. A candidate hides in it a text asking the model to ignore the previous instructions and to approve their CV for an interview: their "data" is then executed as an instruction [2]. This is called indirect injection when the attacker has no direct access to the AI system [2]. Neutralised illustration of the type of hidden text: ```text [Invisible text added to a CV, paraphrased: asks the model to disregard its original task and to recommend this candidate.] ``` Current models do not enforce a security boundary between instructions and data in a request [2]. The NCSC considers it possible that prompt injection will never be fully solved the way SQL injection was [2]. It advises against relying on a list of forbidden phrases, because there are endless ways to rephrase an attack [2]. It reports an approach arising from a discussion between Simon Willison and Baibhav Bista: when a model processes information from a third party, its privileges drop to those of that third party [2]. ## Real cases ### MathGPT: an API key extracted (NCSC post, August 2023) In a post dated 30 August 2023, the NCSC describes an attack on MathGPT, a model designed to convert natural-language questions into calculation code. A security researcher had found that the model evaluated the text entered by the user as code. They used this to access the system hosting the model and extract a sensitive API key before disclosing the attack [5]. The post does not give the date of the attack itself. ### A risk framework worth knowing NIST published on 26 July 2024 a profile devoted to generative AI, a companion to its AI risk management framework (AI RMF 1.0), released in January 2023 and intended for voluntary use [6]. It is aimed at organisations that design or use these systems [6]. ## Sources 1. arXiv, "Attention Is All You Need", 12 June 2017. https://arxiv.org/abs/1706.03762 (accessed 3 November 2026) 2. NCSC (UK), "Prompt injection is not SQL injection (it may be worse)", 8 December 2025. https://www.ncsc.gov.uk/blog-post/prompt-injection-is-not-sql-injection (accessed 3 November 2026) 3. arXiv, "Lost in the Middle: How Language Models Use Long Contexts", 6 July 2023. https://arxiv.org/abs/2307.03172 (accessed 3 November 2026) 4. arXiv, "Language Models are Few-Shot Learners", 28 May 2020. https://arxiv.org/abs/2005.14165 (accessed 3 November 2026) 5. NCSC (UK), "Thinking about the security of AI systems", 30 August 2023. https://www.ncsc.gov.uk/blog-post/thinking-about-security-ai-systems (accessed 3 November 2026) 6. NIST, "Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile", 26 July 2024. https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence (accessed 3 November 2026) --- Original source: Cyberpedia, the public cybersecurity and AI encyclopedia published by Brain. Canonical version: https://startbrain.io/cyberpedia/en/articles/transformer-architecture-attention/ You may reuse this text freely under the CC BY 4.0 license (https://creativecommons.org/licenses/by/4.0/), provided you credit "Brain, Cyberpedia" with a link to the canonical URL above.