Cyberpediaby Brain
ENDEFR

The Transformer and the attention mechanism

The Transformer, described in June 2017: what the attention-based architecture changes for an employee, its limits and prompt injection.

In 30 seconds

Where it happens
Every time a question is put to an AI assistant, whether to summarise a document, write an email or translate.
What to know
The model predicts the most likely next word2. The page draws a piece of advice from this: check what it writes.
The limits
On a long text, it does not use all parts with the same reliability3.
The reflex
Reread and check every answer before using it, and never paste confidential data into a tool the company has not approved.
If it's already done
Tell your manager or the security team, without waiting to see a consequence.

What it is

The Transformer, a model architecture, was first described in a research paper submitted on 12 June 2017 under the title "Attention Is All You Need"1.

Understanding the broad outline of this architecture helps to set how it is used: knowing what the tool really does with a text, why a well-written answer can be wrong, and what is better not to entrust to it.

Straight answers

Where does the Transformer come from?

The founding paper was submitted on 12 June 20171. Its authors propose a network architecture that they describe as « new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely »1. Before it, the dominant models for processing sequences of words were based on so-called recurrent or convolutional networks1.

The reported results concerned machine translation. The model reached 28.4 BLEU on the WMT 2014 English-to-German task, and 41.8 BLEU on the English-to-French task after 3.5 days of training on eight GPUs1. The authors also state that the model is easier to parallelise and needs much less training time1.

What is attention?

Attention is the mechanism that the authors place at the centre of the architecture: the best models of the time already linked the encoder and the decoder through an attention mechanism, and the Transformer relies solely on this principle1. For a non-specialist reader, the useful image is that of a reading in which each word is related to the other words of the text, rather than a strictly word-by-word reading. This image is a simplification: the page does not go into the calculation, and the sources in the file do not detail it.

Does this model understand what you write to it?

The UK National Cyber Security Centre (NCSC) answers on this point in a post dated 8 December 2025. Faced with an instruction, a language model « doesn't understand the text it in the way a person does »: it « is simply predicting the most likely next token from the text so far »2. A fluent answer is therefore a probable sequence of words, not proof of a check.

Why have assistants improved?

A 2020 paper on GPT-3 presents a language model with 175 billion parameters, ten times more than any previous non-sparse model4. Its authors show that increasing model size markedly improves performance on new tasks given with only a few examples4. They also point to datasets where the model still struggles4.

Does the model remember the whole conversation?

Not reliably. A 2023 study, tested on questions covering several documents, finds that performance can degrade markedly when the relevant information changes position in the text provided3. Performance is often better when the information is at the beginning or the end, and degrades when it is in the middle of a long context, including for models designed for long texts3.

Can you entrust it with sensitive data?

No, unless it is in a tool approved by the company. The NCSC lists among the possible effects of prompt injection the ability to « reveal confidential information »5. The page does not detail the rules of each tool: they depend on the company's policy, which the sources in the file do not describe.

1How it works1 min

A Transformer receives a text and produces a sequence of text. The NCSC sums up the principle plainly: the model predicts the « next token », that is, the most likely next element given the text already present2. Nothing in this operation separates what would be an instruction from what would be simple data2.

Two practical consequences follow.

  • The text produced is probable, not checked. A model can write a convincing paragraph without anyone having verified a single fact.
  • There is no security boundary between instructions and data inside a prompt2.

The 2020 paper on GPT-3 also notes that human evaluators had difficulty telling news articles generated by the model from articles written by humans4. The quality of the form therefore says nothing about the accuracy of the substance.

2What you do1 min
  • Reread each answer as a draft written by someone who has checked nothing: figures, names, references and quotations are verified at the source.
  • Performance is often highest when the relevant information is at the beginning or the end of the text provided3. The page infers from this that the core of the request should go in those places rather than in the middle of a long document.
  • For a long exchange or a long document, the page advises starting again with a summary and a precise request rather than assuming that everything has been retained.
  • Do not paste personal data, trade secrets, passwords or access keys. The NCSC reports a case where a security researcher extracted a sensitive API key from a system exposed to a prompt injection attack5.
  • If the assistant reads documents from outside (emails, CVs, web pages), assume that their content may contain hidden instructions2.
3Pitfalls2 min

Believing the model reasons

An answer that sets out steps looks like reasoning. The sources in the file nevertheless describe a mechanism that predicts the next word2, and the NCSC specifies that the model does not understand the text as a person does2.

Taking a fluent answer for an accurate one

A well-turned text can be false. Human evaluators had difficulty telling news articles generated by GPT-3 from articles written by humans4.

Prompt injection

Prompt injection means slipping into a text an instruction that the model will follow. The NCSC for its part defines it as an input designed to make the model behave in an unexpected way: producing shocking content, revealing confidential information or triggering unwanted effects in a system that accepts the model's output without checks5.

The NCSC post of December 2025 gives an example in recruitment. A tool asks the model to rate whether a CV matches the criteria. A candidate hides in it a text asking the model to ignore the previous instructions and to approve their CV for an interview: their "data" is then executed as an instruction2. This is called indirect injection when the attacker has no direct access to the AI system2.

Neutralised illustration of the type of hidden text:

Excerpt
[Invisible text added to a CV, paraphrased: asks the model to disregard
its original task and to recommend this candidate.]

Current models do not enforce a security boundary between instructions and data in a request2. The NCSC considers it possible that prompt injection will never be fully solved the way SQL injection was2. It advises against relying on a list of forbidden phrases, because there are endless ways to rephrase an attack2. It reports an approach arising from a discussion between Simon Willison and Baibhav Bista: when a model processes information from a third party, its privileges drop to those of that third party2.

4Real cases1 min

MathGPT: an API key extracted (NCSC post, August 2023)

In a post dated 30 August 2023, the NCSC describes an attack on MathGPT, a model designed to convert natural-language questions into calculation code. A security researcher had found that the model evaluated the text entered by the user as code. They used this to access the system hosting the model and extract a sensitive API key before disclosing the attack5. The post does not give the date of the attack itself.

A risk framework worth knowing

NIST published on 26 July 2024 a profile devoted to generative AI, a companion to its AI risk management framework (AI RMF 1.0), released in January 2023 and intended for voluntary use6. It is aimed at organisations that design or use these systems6.

Fact sheet

Identifier
CP-257
Wing
4. Artificial intelligence
Brain skill
Understanding AI & LLMs (understand_ai)
Brain reflex
air-038
Diagram
G3 · System & Protocol Anatomy
Languages
de, en, fr
Sources
6 sources checked on 3 November 2026
Sources6 sources
  1. arXiv, “Attention Is All You Need”, 12 June 2017. https://arxiv.org/abs/1706.03762 (accessed 3 November 2026)
  2. NCSC (UK), “Prompt injection is not SQL injection (it may be worse)”, 8 December 2025. https://www.ncsc.gov.uk/blog-post/prompt-injection-is-not-sql-injection (accessed 3 November 2026)
  3. arXiv, “Lost in the Middle: How Language Models Use Long Contexts”, 6 July 2023. https://arxiv.org/abs/2307.03172 (accessed 3 November 2026)
  4. arXiv, “Language Models are Few-Shot Learners”, 28 May 2020. https://arxiv.org/abs/2005.14165 (accessed 3 November 2026)
  5. NCSC (UK), “Thinking about the security of AI systems”, 30 August 2023. https://www.ncsc.gov.uk/blog-post/thinking-about-security-ai-systems (accessed 3 November 2026)
  6. NIST, “Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile”, 26 July 2024. https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence (accessed 3 November 2026)

Cite this article

Brain, “The Transformer and the attention mechanism”, Cyberpedia, 2026. https://startbrain.io/cyberpedia/en/articles/transformer-architecture-attention/

BibTeX 
@misc{brain_cp257_en_2026,
  author = {Brain},
  title = {The Transformer and the attention mechanism},
  year = {2026},
  howpublished = {Cyberpedia},
  url = {https://startbrain.io/cyberpedia/en/articles/transformer-architecture-attention/},
  note = {CC BY 4.0}
}
CC BY 4.0 licence