--- id: CP-269 slug: on-device-ai-npu-vs-cloud-ai lang: en title: On-device AI or cloud AI description: "On-device AI (NPU chip) and cloud AI raise different questions: choosing an approved tool, what to enter, and whom to tell." canonical_url: https://startbrain.io/cyberpedia/en/articles/on-device-ai-npu-vs-cloud-ai/ markdown_url: https://startbrain.io/cyberpedia/en/articles/on-device-ai-npu-vs-cloud-ai.md publisher: Brain (https://startbrain.io) license: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) cite_as: Brain, "On-device AI or cloud AI", Cyberpedia, 2026. https://startbrain.io/cyberpedia/en/articles/on-device-ai-npu-vs-cloud-ai/ (CC BY 4.0) wing: 4 theme: authorized_tools reflex: air-021 schema: G3 updated: "2026-11-03" translations: en: https://startbrain.io/cyberpedia/en/articles/on-device-ai-npu-vs-cloud-ai/ fr: https://startbrain.io/cyberpedia/fr/articles/ia-locale-npu-vs-ia-cloud/ de: https://startbrain.io/cyberpedia/de/articles/lokale-ki-npu-vs-cloud-ki/ es: https://startbrain.io/cyberpedia/es/articles/ia-local-npu-vs-ia-nube/ sources: - "n": 1 title: Develop AI applications for Copilot+ PCs publisher: Microsoft url: https://learn.microsoft.com/en-us/windows/ai/npu-devices/ accessed: "2026-11-03" - "n": 2 title: Guidelines for secure AI system development publisher: NCSC (UK) url: https://www.ncsc.gov.uk/collection/guidelines-secure-ai-system-development published: "2023-11-27" accessed: "2026-11-03" - "n": 3 title: LLMRisks Archive publisher: OWASP url: https://genai.owasp.org/llm-top-10/ accessed: "2026-11-03" - "n": 4 title: "AI Data Security: Best Practices for Securing Data Used to Train & Operate AI Systems" publisher: CISA url: https://www.cisa.gov/resources-tools/resources/ai-data-security-best-practices-securing-data-used-train-operate-ai-systems published: "2025-05-22" accessed: "2026-11-03" - "n": 5 title: Les fiches pratiques IA publisher: CNIL url: https://www.cnil.fr/fr/les-fiches-pratiques-ia accessed: "2026-11-03" - "n": 6 title: Intelligence artificielle publisher: CNIL url: https://www.cnil.fr/fr/intelligence-artificielle accessed: "2026-11-03" - "n": 7 title: "Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations" publisher: NIST url: https://csrc.nist.gov/pubs/ai/100/2/e2023/final accessed: "2026-11-03" - "n": 8 title: AI Risk Management Framework publisher: NIST url: https://www.nist.gov/itl/ai-risk-management-framework accessed: "2026-11-03" --- # On-device AI or cloud AI A "local" AI does its calculations on the employee's computer. A "cloud" AI does them on servers hosted elsewhere, to which the tool connects. To someone typing a question or pasting a document, the two look alike. The risks to company data are nonetheless not the same, and the decision to use a given tool belongs to the company. This article explains the difference, then what an employee can do: choose an approved tool, know what can be entered into it, and ask when in doubt. ## In 30 seconds - Where it happens: translation or image-generation assistants, on a recent computer fitted with an AI chip [1]. - What changes: a local AI runs on the machine, a cloud AI on a hosted service. In both cases, the data entered and the tool itself must be kept under control. - The reflex: use only the tools approved by the company, and ask security or the IT department before entering sensitive data into a tool or installing a new one. - If it has already happened: tell the IT department without delay, stating the tool and what was entered. ## Straight answers ### What is an NPU chip? Microsoft describes it as a chip specialised in heavy AI workloads, such as real-time translation or image generation [1]. Microsoft's "Copilot+" PCs include one capable of more than 40 trillion operations per second (40 TOPS) [1]. The NPU works alongside the processor and the graphics card, and Windows 11 distributes tasks among them [1]. It enables AI functions to run on the device [1]. ### Is a local AI risk-free? No. Nothing in the sources cited here says that a model running on the computer removes the need for precautions. The UK's NCSC points out that AI systems are exposed to new vulnerabilities, to be considered in addition to ordinary IT threats [2]. In its top ten risks for applications built on language models, OWASP lists sensitive information disclosure, the supply chain, and data and model poisoning [3]. None of these sources limits these risks to tools hosted elsewhere. ### What does cloud AI change? The NCSC aims its guidelines mainly at providers of AI systems that use models hosted by an organisation or external application programming interfaces (APIs) [2]. With an external API, the tool depends on a third-party service. The sources cited here do not detail what each provider does with the data entered: that point is found in the terms of service and is checked with the company's IT department. ### What can I enter into an AI tool? The sources cited here set no list of permitted data. The applicable rule is the company's. CISA links data security to the accuracy, integrity and reliability of an AI's results, at every stage of its life [4]. When in doubt, it is better to enter nothing confidential or personal until you have an answer. ### Whom should I ask when in doubt? The company's IT department or security team, and the data protection officer if personal data is involved. These contacts vary from one company to another, and no source cited here names them. ## How it works An AI model runs (this is called "inference") on hardware. On a PC with an NPU, Microsoft recommends going through Windows ML to run AI tasks on the NPU [1]. If the NPU is not available, Windows ML falls back to the GPU or the processor [1]. Windows ML relies on ONNX Runtime, an open-source inference engine [1]. Models are often published in heavy formats. Many NPUs accept only reduced-precision integer calculations, so models must be converted ("quantized") before they run on them [1]. Some already-converted models exist, and anyone can bring their own model to convert [1]. This last point matters for the employee: a local model is a file, and it comes from someone. ## What you do - Use the tools provided or approved by the company, whether local or in the cloud. - Before installing a tool or a model, ask the IT department for approval. - Before entering a sensitive document or people's data, ask whether the tool is authorised for that use. The CNIL publishes practical guides on applying the GDPR to AI, including one on the security of AI system development [5]. - If you have already entered data into an unapproved tool, report it. ## Pitfalls ### Believing a local AI is risk-free The calculation happens on the computer, but the model and the application remain software that must be kept up to date. The NCSC lists updates and logging among the actions of the operation and maintenance phase of an AI system [2]. OWASP also flags the risk of "excessive agency": a system built on a language model is often given some autonomy [3]. A local tool connected to your files or your mailboxes is one example to examine with the IT department. ### Using an unapproved tool ("shadow AI") An unapproved tool escapes the company's checks. The NCSC recommends treating security as a basic requirement, across a system's whole life cycle [2]. A tool chosen alone by an employee falls under none of these controls. ### Downloading a model of unknown origin A downloaded model is an element of the supply chain, a risk that OWASP cites explicitly [3]. The NIST report on adversarial machine learning deals with backdoor and "Trojan" attacks, as well as data poisoning [7]. For its part, the CNIL maintains a tool for exploring the genealogy of AI models published as open source, updated on 26 August 2026 [6]. For a model of unknown origin, ask the IT department for approval before installing it. ### Neglecting data access An AI that reads documents can reveal their content. OWASP cites sensitive information disclosure, which affects both the model and its application [3]. The NIST risk management framework, voluntary in use, aims to build trustworthiness into the design, use and evaluation of AI systems [8]. ## Sources 1. Microsoft, "Develop AI applications for Copilot+ PCs". https://learn.microsoft.com/en-us/windows/ai/npu-devices/ (accessed 3 November 2026) 2. NCSC (UK), "Guidelines for secure AI system development", 27 November 2023. https://www.ncsc.gov.uk/collection/guidelines-secure-ai-system-development (accessed 3 November 2026) 3. OWASP, "LLMRisks Archive". https://genai.owasp.org/llm-top-10/ (accessed 3 November 2026) 4. CISA, "AI Data Security: Best Practices for Securing Data Used to Train & Operate AI Systems", 22 May 2025. https://www.cisa.gov/resources-tools/resources/ai-data-security-best-practices-securing-data-used-train-operate-ai-systems (accessed 3 November 2026) 5. CNIL, "Les fiches pratiques IA". https://www.cnil.fr/fr/les-fiches-pratiques-ia (accessed 3 November 2026) 6. CNIL, "Intelligence artificielle". https://www.cnil.fr/fr/intelligence-artificielle (accessed 3 November 2026) 7. NIST, "Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations". https://csrc.nist.gov/pubs/ai/100/2/e2023/final (accessed 3 November 2026) 8. NIST, "AI Risk Management Framework". https://www.nist.gov/itl/ai-risk-management-framework (accessed 3 November 2026) --- Original source: Cyberpedia, the public cybersecurity and AI encyclopedia published by Brain. Canonical version: https://startbrain.io/cyberpedia/en/articles/on-device-ai-npu-vs-cloud-ai/ You may reuse this text freely under the CC BY 4.0 license (https://creativecommons.org/licenses/by/4.0/), provided you credit "Brain, Cyberpedia" with a link to the canonical URL above.