AI model theft through API queries
How a competitor copies an AI model by querying its API, and what an employee of a company that exposes one should take from it.
On this page
In 30 seconds
- Where it happens
- On the API of a model offered as a service. The attacks Google describes targeted its Gemini models[1].
- What the attacker wants
- To speed up the development of their own model, quickly and at a much lower cost, by reusing what a mature model has learned[1].
- Who is affected
- Model providers and AI service providers. Ordinary users are typically not exposed to a risk[1].
What it is
An AI model offered as a service can be copied without any server being hacked. According to the Google Threat Intelligence Group (GTIG), actors use legitimate API access to try to clone some of a model's capabilities, where stealing trade secrets once required a classic computer intrusion[1]. The principle has two steps: query the model methodically, then train another model on its answers[1].
This entry is aimed first at model providers. Google writes that other companies with AI models will probably face this in the near future[2], and expects these attacks to appear elsewhere as more organizations expose their models to customers and the public[3]. An employee of a company that offers a model to its customers is therefore concerned by API monitoring and access protection.
Straight answers
What is a model extraction attack?
It happens when an adversary uses legitimate access to probe a mature machine learning model methodically, in order to draw out information used to train a new model[1]. To do this, the attacker relies on knowledge distillation, which is why these are also called "distillation attacks"[1].
Is distillation always an attack?
No. It is a common technique for training a "student" model from an existing "teacher" model[1]. It often consists of querying the teacher on problems in a specific domain, then fine-tuning the student model with the answers (supervised fine-tuning)[1]. There are legitimate uses, and Google Cloud offers distillation products[1]. However, distilling Gemini models without authorization violates Google's terms of service[1].
Is an ordinary user at risk?
Google states that these attacks typically do not pose a risk to ordinary users, because they threaten neither the confidentiality, nor the availability, nor the integrity of AI services[1]. The risk is concentrated among model developers and service providers[1].
Who carries out these attacks?
Google says it has observed and mitigated frequent extraction attacks from private-sector entities around the world and from researchers seeking to clone proprietary logic[1]. In 2025, the group observed no direct attacks on frontier models from the state-backed actors or influence operations it tracks[1].
What should a company that exposes a model to its customers do?
Google recommends that organizations providing AI models as a service monitor API access for extraction or distillation patterns[1]. OWASP adds other measures: rate limiting and per-source quotas, logging usage and detecting anomalies, and strict access control[4]. They are detailed below.
Inside the attacker's head
The attacker is looking to save time and money. Extraction followed by distillation lets them speed up the development of a model quickly and at a much lower cost, which Google calls intellectual property theft[1].
The target is the proprietary logic and specialized training of a model[1]. Google gives two examples: a model customized for financial data analysis, targeted by a commercial competitor who wants to build a derivative product, and a coding model, targeted by an adversary who wants to reproduce its capabilities in an environment without safeguards[1].
Attackers also seek to obtain Gemini's reasoning capability, a frequent target[1]. OWASP describes a variant, functional replication: the attacker uses the targeted model to generate synthetic training data, then fine-tunes another model to obtain a functional equivalent[4].
What you see on screen
For an ordinary user, these attacks typically affect neither the confidentiality, nor the availability, nor the integrity of the service[1]. The attacker goes through legitimate API access[1]. This is why Google recommends that providers monitor this access to spot extraction patterns[1], and OWASP recommends continuously monitoring resource use and logging to detect unusual patterns[4].
Under the hoodIT teams
The mechanism follows that of legitimate distillation. The attacker sends the teacher model questions on a domain, collects the answers, then uses them to train their student model[1].
Gemini's internal reasoning traces are normally summarized before being delivered to the user. Attackers tried to force the model to output its full reasoning processes[1].
OWASP classifies model extraction through the API among the forms of unbounded consumption: the attacker queries the API with carefully crafted inputs and prompt injection techniques, to collect enough outputs to replicate a partial model or create a "shadow model"[4]. Beyond copying, OWASP notes that extraction undermines the integrity of the original model[4].
The defense measures OWASP names for this risk are the following[4]:
- limit the size of inputs;
- restrict or mask the
logit_biasandlogprobsparameters in API responses; - apply rate limiting and per-source usage quotas;
- log and monitor to detect unusual consumption;
- embed a watermark in model outputs to detect unauthorized use;
- train models to detect adversarial queries and extraction attempts;
- put strong access controls in place (RBAC, principle of least privilege) to limit unauthorized access to model repositories and training environments.
For its part, Google says it uses proactive real-time defenses, which may degrade the performance of the student model, and points out that these attacks may be subject to takedowns and legal action[1].
Access itself is a target. To abuse AI services at scale, malicious actors need API keys, which creates a hijacking risk for organizations with many cloud and AI resources[1]. Vulnerable open-source AI tools are commonly exploited to steal API keys, which feeds a black market for reselling access and places costs on the affected users[1]. OWASP points out that the training methods and source code of proprietary models may be considered sensitive[5]. In addition, poorly configured model outputs may reveal proprietary algorithms or data[5]. The "Proof Pudding" attack (CVE-2019-20634) is one example: leaked training data facilitated model extraction and inversion, and allowed email filters to be bypassed[5].
Case filesReal cases
February 12, 2026: a campaign of more than 100,000 requests against Gemini
On February 12, 2026, the GTIG published a report describing extraction attacks against Gemini models, identified and disrupted by Google DeepMind and the GTIG[1]. They came from researchers and private companies around the world[1].
One detailed case targeted Gemini's reasoning. The attackers asked the model to output its full reasoning, requiring that the language of the thinking content stay consistent with the primary language of the input[1]. The analysis counted more than 100,000 requests (prompts)[1]. According to the GTIG, the breadth of the questions suggests an attempt to replicate Gemini's reasoning capability in languages other than English, across a wide variety of tasks[1].
Google's systems recognized the attack in real time and reduced its risk, which protected the internal reasoning traces[1].
Sources
- Google, “GTIG AI Threat Tracker: Distillation, Experimentation, and (Continued) Integration of AI for Adversarial Use”, 12 February 2026. https://cloud.google.com/blog/topics/threat-intelligence/distillation-experimentation-integration-ai-adversarial-use (accessed 14 November 2026)
- Google, “Our new report details the latest ways threat actors are misusing AI.”, 12 February 2026. https://blog.google/innovation-and-ai/infrastructure-and-cloud/google-cloud/gtig-report-ai-cyber-attacks-feb-2026/ (accessed 14 November 2026)
- Google, “Cloud CISO Perspectives: New AI threats report: Distillation, experimentation, and integration”, 19 February 2026. https://cloud.google.com/blog/products/identity-security/cloud-ciso-perspectives-new-ai-threats-report-distillation-experimentation-integration (accessed 14 November 2026)
- OWASP, “LLM10:2025 Unbounded Consumption”, 10 April 2024. https://genai.owasp.org/llmrisk/llm102025-unbounded-consumption/ (accessed 14 November 2026)
- OWASP, “LLM02:2025 Sensitive Information Disclosure”, 10 April 2024. https://genai.owasp.org/llmrisk/llm022025-sensitive-information-disclosure/ (accessed 14 November 2026)
Cite this article
Brain, “AI model theft through API queries”, Cyberpedia, 2026. https:/
BibTeX
@misc{brain_cp290_en_2026,
author = {Brain},
title = {AI model theft through API queries},
year = {2026},
howpublished = {Cyberpedia},
url = {https://startbrain.io/cyberpedia/en/articles/model-extraction-ai-model-theft/},
note = {CC BY 4.0}
}