Adversarial examples against vision AI
A near-identical image or a simple sticker can fool a vision model. How the principle works, dated cases and defenses.
On this page
In 30 seconds
- Where it happens
- On systems that read images, notably those that receive their data from a camera or another sensor[1].
- What the attacker wants
- To get a specific answer of their choosing, for example to have a sign classified in another category[3].
- The reflex
- Do not take for granted a machine's reading of an image, and report to the team responsible for the system any absurd decision made while the image looks normal.
- If it has already happened
- Keep the image or the sequence that caused the error and pass it to that team.
What it is
An adversarial example is an input that has been modified very slightly, with the aim of pushing a machine learning model into making a mistake[1]. For an image, the modification can be so discreet that a human observer does not notice it, yet the model still gets it wrong[1].
For an employee, that is the difficulty: the image looks normal, and it is the machine that decides otherwise. As early as 2013, researchers showed that an imperceptible perturbation is enough to make a neural network put an image in the wrong category[2].
Straight answers
Can you see that an image has been modified?
Not always. In many cases, the modifications are so subtle that a human does not notice them, while the classifier gets it wrong[1]. In the case of road signs, the perturbation takes the form of black and white stickers placed on a real stop sign[3].
Do you need access to the model to fool it?
Not necessarily. The same perturbation can make another network, trained on a different part of the data, go wrong[2]. The 2016 authors write that these examples pose a security problem because they could be used to attack machine learning systems « even if the adversary has no access to the underlying model »[1].
Does a camera in the real world count, or only files sent to a program?
It does. Adversarial images obtained with a phone camera were shown to an Inception classifier on ImageNet: a large fraction of them remains misclassified after passing through the camera[1].
Do defenses solve the problem?
A defense can be bypassed. Defensive distillation had been presented as lowering the success rate of attacks from 95% to 0.5%, then three new attack algorithms bypassed it with a probability of 100%[6].
Inside the attacker's head
The attacker looks for an image that resembles the original but triggers a wrong answer. The 2013 researchers find it by maximizing the network's prediction error[2]. In 2014, Goodfellow, Shlens and Szegedy describe a simple and fast method for producing such examples, based on their explanation of the vulnerability: the main cause would be the linear nature of neural networks[4].
The attacker has several possible goals. In the case of signs, the aim is a specific classification, known as targeted[3]. With an adversarial patch, the attacker can make the classifier output any target class of their choosing[5]. The same perturbation can fool another network[2], and these examples could be used to attack a system even if the adversary has no access to the model[1].
What you see on screen
On screen or on the camera, the scene looks ordinary. A human observer sometimes notices no modification at all, and yet the classifier gets it wrong[1]. In the case of an adversarial patch, the printed pattern is added to the scene, photographed, then shown to the classifier, which then ignores the other elements of the scene and reports the class chosen by the attacker, even when the patch is small[5].
The warning sign is therefore the decision, not the image: an answer from the system that has nothing to do with what a person sees in the same scene.
Under the hoodIT teams
A neural network learns mappings between inputs and outputs that are, according to Szegedy and his colleagues, discontinuous to a significant extent[2]. An imperceptible perturbation, found by maximizing the network's error, is then enough to change the predicted class, and it is not a random artifact of training[2].
Goodfellow and his coauthors set aside the earlier explanations, non-linearity and overfitting, in favor of the linear nature of networks. This reading, they say, also explains why examples generalize from one architecture and one training set to another[4]. The model then gets it wrong with high confidence[4].
For the physical world, Eykholt and his coauthors propose an algorithm named Robust Physical Perturbations (RP2), which produces visual perturbations that are robust under different physical conditions, including viewpoints[3]. The patch by Brown and his coauthors is universal (it attacks any scene), robust (it withstands many transformations) and targeted[5].
Case filesReal cases
December 2013: imperceptible perturbations that carry over from one network to another
In December 2013, Szegedy and six coauthors publish "Intriguing properties of neural networks". They show that an imperceptible perturbation found by maximizing the prediction error makes the image be misclassified, and that the same perturbation fools another network trained on a different subset of the data[2].
July 2016: a phone camera does not neutralize the attack
In July 2016, Kurakin, Goodfellow and Bengio point out that earlier work assumed an adversary able to send data directly to the classifier, which is not the case for systems that receive signals from cameras[1]. Adversarial images passed through a phone camera remain largely misclassified[1].
August 2016: defensive distillation bypassed
Carlini and Wagner publish in August 2016 a piece of work that challenges defensive distillation. Their three new algorithms succeed with a probability of 100% on distilled and undistilled networks alike, and they propose a high-confidence transferability test that also defeats it[6].
July 2017: the stop sign and its stickers
In July 2017, Eykholt and eight coauthors publish their attack on road sign classification. With black and white stickers only, they attack a real stop sign. The targeted error affects 100% of the images taken in the lab and 84.8% of the video images captured from a moving vehicle during the field test, for the targeted classifier[3]. They also propose a two-stage evaluation method, lab then field, for lack of a standardized test method[3].
December 2017: the printable adversarial patch
In December 2017, Brown, Mané, Roy, Abadi and Gilmer describe a patch that can be printed, added to a scene, photographed and shown to a classifier[5].
What you do
- If you use or monitor a system that reads images, treat its output as a proposal to be checked when it contradicts what you see.
- Report an absurd decision to the team that runs the system, with the image or the sequence concerned.
- Do not remove or clean off a suspicious sticker or object without having told that team: it serves as evidence.
Pitfalls
- A "normal" image is not a safe image: the attack can be invisible to the eye[1].
- A single defense is not enough: defensive distillation was bypassed[6].
- The attacker does not necessarily need access to the model: the 2016 authors note that these examples could be used to attack a system without that access[1].
Defenses and reference frameworks
Adversarial training consists of using adversarial examples during learning. Goodfellow and his coauthors used it to reduce the error of a maxout network on the MNIST test set[4].
NIST publishes report AI 100-2 E2023, final version of January 4, 2024, which classifies attacks against machine learning, defines a vocabulary, presents mitigation methods and points out challenges that remain open[7]. The British NCSC published on May 22, 2024 security principles for machine learning. These principles are neither a complete evaluation framework nor a checklist, and they ask that security be treated as a requirement throughout the system's life cycle[8]. For its part, OWASP is preparing a Machine Learning Security Top Ten, still at the draft stage[9].
Sources
- arXiv, “Adversarial examples in the physical world”, 8 July 2016. https://arxiv.org/abs/1607.02533 (accessed 10 November 2026)
- arXiv, “Intriguing properties of neural networks”, 21 December 2013. https://arxiv.org/abs/1312.6199 (accessed 10 November 2026)
- arXiv, “Robust Physical-World Attacks on Deep Learning Models”, 27 July 2017. https://arxiv.org/abs/1707.08945 (accessed 10 November 2026)
- arXiv, “Explaining and Harnessing Adversarial Examples”, 20 December 2014. https://arxiv.org/abs/1412.6572 (accessed 10 November 2026)
- arXiv, “Adversarial Patch”, 27 December 2017. https://arxiv.org/abs/1712.09665 (accessed 10 November 2026)
- arXiv, “Towards Evaluating the Robustness of Neural Networks”, 16 August 2016. https://arxiv.org/abs/1608.04644 (accessed 10 November 2026)
- NIST, “Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations”, 4 January 2024. https://csrc.nist.gov/pubs/ai/100/2/e2023/final (accessed 10 November 2026)
- NCSC (UK), “Machine learning principles”, 22 May 2024. https://www.ncsc.gov.uk/collection/machine-learning-principles (accessed 10 November 2026)
- OWASP, “OWASP Machine Learning Security Top Ten”. https://owasp.org/projects/machine-learning-security-top-ten (accessed 10 November 2026)
Cite this article
Brain, “Adversarial examples against vision AI”, Cyberpedia, 2026. https:/
BibTeX
@misc{brain_cp289_en_2026,
author = {Brain},
title = {Adversarial examples against vision AI},
year = {2026},
howpublished = {Cyberpedia},
url = {https://startbrain.io/cyberpedia/en/articles/adversarial-examples-vision-models/},
note = {CC BY 4.0}
}