---
id: CP-292
slug: poisoned-ai-models-pickle-files
lang: en
title: "Poisoned AI models: the risk of Pickle files"
description: "Downloading an AI model from a sharing site is not harmless: the pickle format exposes you to malicious models. Key points and steps."
canonical_url: https://startbrain.io/cyberpedia/en/articles/poisoned-ai-models-pickle-files/
markdown_url: https://startbrain.io/cyberpedia/en/articles/poisoned-ai-models-pickle-files.md
publisher: Brain (https://startbrain.io)
license: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)
cite_as: "Brain, \"Poisoned AI models: the risk of Pickle files\", Cyberpedia, 2026. https://startbrain.io/cyberpedia/en/articles/poisoned-ai-models-pickle-files/ (CC BY 4.0)"
wing: 4
theme: authorized_tools
reflex: air-018
schema: G1
updated: "2026-11-14"
translations:
  en: https://startbrain.io/cyberpedia/en/articles/poisoned-ai-models-pickle-files/
  fr: https://startbrain.io/cyberpedia/fr/articles/modeles-ia-pieges-fichiers-pickle/
  de: https://startbrain.io/cyberpedia/de/articles/praeparierte-ki-modelle-pickle-dateien/
  es: https://startbrain.io/cyberpedia/es/articles/modelos-ia-con-trampa-archivos-pickle/
  it: https://startbrain.io/cyberpedia/it/articles/modelli-ia-con-trappola-file-pickle/
  nl: https://startbrain.io/cyberpedia/nl/articles/ai-modellen-met-valstrik-pickle-bestanden/
  pl: https://startbrain.io/cyberpedia/pl/articles/modele-ai-z-pulapka-pliki-pickle/
  pt: https://startbrain.io/cyberpedia/pt/articles/modelos-ia-armadilhados-ficheiros-pickle/
  ja: https://startbrain.io/cyberpedia/ja/articles/poisoned-ai-models-pickle-files/
sources:
  - "n": 1
    title: LLM03:2025 Supply Chain
    publisher: OWASP
    url: https://genai.owasp.org/llmrisk/llm032025-supply-chain/
    published: "2024-04-10"
    accessed: "2026-11-14"
  - "n": 2
    title: "Models Are Codes: Towards Measuring Malicious Code Poisoning Attacks on Pre-trained Model Hubs"
    publisher: arXiv
    url: https://arxiv.org/abs/2409.09368
    published: "2024-09-14"
    accessed: "2026-11-14"
  - "n": 3
    title: "The Art of Hide and Seek: Making Pickle-Based Model Supply Chain Poisoning Stealthy Again"
    publisher: arXiv
    url: https://arxiv.org/abs/2508.19774
    published: "2025-08-27"
    accessed: "2026-11-14"
  - "n": 4
    title: "PickleBall: Secure Deserialization of Pickle-based Machine Learning Models (Extended Report)"
    publisher: arXiv
    url: https://arxiv.org/abs/2508.15987
    published: "2025-08-21"
    accessed: "2026-11-14"
  - "n": 5
    title: An Empirical Study on Remote Code Execution in Machine Learning Model Hosting Ecosystems
    publisher: arXiv
    url: https://arxiv.org/abs/2601.14163
    published: "2026-01-20"
    accessed: "2026-11-14"
  - "n": 6
    title: AI Risk Management Framework
    publisher: NIST
    url: https://www.nist.gov/itl/ai-risk-management-framework
    published: "2021-07-12"
    accessed: "2026-11-14"
---

# Poisoned AI models: the risk of Pickle files

A pre-trained AI model is downloaded from a sharing platform like any other file. According to OWASP, third-party pre-trained models are part of the supply chain of AI applications, and these external components can be manipulated through tampering or poisoning [1].

For some file formats, the risk is more concrete. In 2024, researchers reported 91 malicious models on a Hugging Face mirror [2]. In 2025, others showed that scanners for pickle-format models could be bypassed [3]. For a malicious pickle model, the threat described by researchers is the calling of arbitrary functions at load time [4].

## In 30 seconds

- Where it happens: on model-sharing platforms such as Hugging Face, where malicious actors can deliver malware through compromised models [4].
- What the attacker wants: to act on the machine that loads the model. The 91 malicious models and the 9 malicious dataset-loading scripts spotted in 2024 reveal threats such as reverse shell, browser credential theft and system reconnaissance [2].
- The reflex: do not load a model whose source you cannot verify. OWASP recommends using only models from verifiable sources [1].

## Straight answers

### Why can a model file be dangerous?

The pickle format saves and restores complex objects as a stream of bytes. According to the authors of a 2025 study, many AI frameworks still adopt it for models despite its own risks [3]. Pickle deserialization flaws have long been known in Python without being resolved [3]. The PickleBall authors note that 44.9% of popular Hugging Face models still use this format, which they consider unsafe [4].

### What did researchers find in 2024?

A team deployed its MalHug tool on a Hugging Face mirror hosted at Ant Group. It monitored more than 705,000 models and 176,000 datasets there, and spotted 91 malicious models and 9 malicious dataset-loading scripts [2].

### Are analysis tools enough?

Existing tools such as Fickling and ModelScan offer some protection, but according to the MalHug authors they have limits in semantic analysis and in complete threat detection [2]. In 2025, researchers identified 22 pickle model loading paths in five AI frameworks, 19 of which entirely escape existing scanners [3]. Other authors note that model scanners produce false positives and false negatives [4].

### Is the format the only problem?

A January 2026 study looks at the execution of untrusted code while a model is loading, when the model comes with custom code. Its authors find widespread reliance on unsafe default settings, uneven security enforcement across platforms, and persistent confusion among developers about the consequences of remote code execution [5].

### What steps limit the risk?

For models supplied by third parties, OWASP recommends the following measures [1]:

- carefully review data sources and suppliers, including their terms of use and privacy policies, and use only trusted suppliers;
- use only models from verifiable sources, and rely on third-party integrity checks with signing and file hashes, to make up for the lack of strong guarantees on provenance;
- apply comprehensive red teaming and evaluations when choosing a third-party model;
- use anomaly detection and robustness tests, which can help spot tampering, even though these techniques are still emerging.

## Inside the attacker's head

According to OWASP, there are currently no strong guarantees on the provenance of published models. An attacker can compromise a supplier's account on a model repository, or create a look-alike account, then add social engineering to reach an AI application [1].

OWASP gives two scenarios. In the first, after the WizardLM model was withdrawn, an attacker takes advantage of the interest it had generated and publishes a fake version under the same name, with malware and backdoors [1]. In the second, an attacker targets a model merging or conversion service to inject malware into a publicly available model. OWASP states that this is a real attack published by the vendor HiddenLayer [1].

On the research side, the authors of the 2025 study write that the poisoning of pickle models can remain stealthy [3].

## What you see on screen

OWASP notes that model cards and the associated documentation provide information about the model, but offer no guarantee about its origin [1]. The fake version of WizardLM carried the same name as the original [1].

Public scores guarantee nothing either. OWASP states that models can be fine-tuned to get around published benchmarks, and describes a scenario in which an attacker uploads to Hugging Face a model tuned to obtain good safety scores, exploiting victims' trust in those assurances [1].

Three markers follow: a source that cannot be verified, a provenance that is not established [1], and a file format for which researchers describe the threat of arbitrary function calls at load time [4].

## Under the hood

According to the authors of a 2025 study, pickle makes it possible to save and restore complex objects as bytes, which explains its adoption by many AI frameworks [3]. The same authors also describe the resulting poisoning surface, on both the model loading side and the risky functions side [3].

On the defense side, the MalHug team combines the extraction of dataset-loading scripts, model deserialization, taint analysis and heuristic pattern matching to detect and classify attacks [2]. The PickleBall team analyzes the source code of an AI library to compute a safe loading policy, then enforces it during loading as a drop-in replacement for the pickle module [4]. In their dataset, this policy correctly loads 79.8% of benign pickle models and rejects 100% of malicious examples [4]. The same authors specify that 15% of popular pickle models cannot be loaded with restrictive loading policies [4].

NIST, for its part, offers the AI RMF framework, intended for voluntary use, to better incorporate trustworthiness into the design, development, use and evaluation of AI products [6]. Its generative AI profile, published on July 26, 2024, can help organizations identify the risks specific to generative AI [6].

## Case files

### 2024: 91 malicious models on a Hugging Face mirror

In a paper submitted on September 14, 2024, researchers report having deployed MalHug, in collaboration with Ant Group, on a Hugging Face mirror within the company's infrastructure. The tool remained operational for more than three months, monitored more than 705,000 models and 176,000 datasets, and spotted 91 malicious models and 9 malicious dataset-loading scripts [2]. The threats identified include reverse shell, browser credential theft and system reconnaissance [2].

### 2025: model scanners bypassed

In a paper submitted on August 27, 2025, researchers show that scanners designed to detect poisoned pickle models have fragile detection logic, because their understanding of the poisoning surface is incomplete [3]. Of 22 loading paths identified, 19 entirely escape existing scanners [3]. The researchers also found 9 instances of a bypass technique they call Exception-Oriented Programming, 7 of which bypass all scanners [3]. They found 133 exploitable gadgets, with a bypass rate close to 100%, and 89% against the best scanner [3]. This is a flaw demonstrated by researchers. They responsibly disclosed their findings to the vendors concerned and received acknowledgments as well as a 6,000 dollar bounty [3].

### 2026: code execution at load time, a large-scale study

In a paper submitted on January 20, 2026, researchers study model-loading practices with custom code on five sharing platforms, including Hugging Face, ModelScope and OpenCSG [5]. They also analyze more than 600 developer discussions on GitHub, the Hugging Face and PyTorch Hub forums, and Stack Overflow [5].

## Sources

1. OWASP, "LLM03:2025 Supply Chain", 10 April 2024. https://genai.owasp.org/llmrisk/llm032025-supply-chain/ (accessed 14 November 2026)
2. arXiv, "Models Are Codes: Towards Measuring Malicious Code Poisoning Attacks on Pre-trained Model Hubs", 14 September 2024. https://arxiv.org/abs/2409.09368 (accessed 14 November 2026)
3. arXiv, "The Art of Hide and Seek: Making Pickle-Based Model Supply Chain Poisoning Stealthy Again", 27 August 2025. https://arxiv.org/abs/2508.19774 (accessed 14 November 2026)
4. arXiv, "PickleBall: Secure Deserialization of Pickle-based Machine Learning Models (Extended Report)", 21 August 2025. https://arxiv.org/abs/2508.15987 (accessed 14 November 2026)
5. arXiv, "An Empirical Study on Remote Code Execution in Machine Learning Model Hosting Ecosystems", 20 January 2026. https://arxiv.org/abs/2601.14163 (accessed 14 November 2026)
6. NIST, "AI Risk Management Framework", 12 July 2021. https://www.nist.gov/itl/ai-risk-management-framework (accessed 14 November 2026)

---

Original source: Cyberpedia, the public cybersecurity and AI encyclopedia published by Brain. Canonical version: https://startbrain.io/cyberpedia/en/articles/poisoned-ai-models-pickle-files/
You may reuse this text freely under the CC BY 4.0 license (https://creativecommons.org/licenses/by/4.0/), provided you credit "Brain, Cyberpedia" with a link to the canonical URL above.
