AI Security

AI Supply Chain Security: Models, Data, Code, and Provenance

An operational AI supply-chain guide covering models, datasets, code, containers, prompts, services, provenance, release controls, and incident response.

By PermsAI Editorial Team

AI supply chain security protects the models, data, code, configuration, infrastructure, and external services that an AI system depends on from selection through retirement. The deployable product is rarely one model file. It is a graph of artifacts and suppliers whose origin, integrity, permissions, and versions determine what actually runs.

A practical program must answer two questions at any time: “What exactly is deployed?” and “What evidence justifies trusting each component?” Hashes, signatures, bills of materials, controlled builds, dataset lineage, vendor records, evaluations, and runtime telemetry provide different parts of that answer. None alone proves an AI system is safe.

What the AI supply chain includes

Inventory the components that can change behavior, access, or execution:

base model weights and tokenizer files;
hosted model APIs and selected versions;
pretraining, fine-tuning, evaluation, and retrieval datasets;
adapters such as LoRA artifacts, embeddings, and model conversions;
Python, JavaScript, native, GPU, and orchestration packages;
container images, drivers, inference runtimes, and serving configuration;
system prompts, policy files, templates, safety settings, and feature flags;
tools, plugins, vector databases, external APIs, and identity providers;
source repositories, build systems, registries, deployment pipelines, and backups.

NIST SP 800-218A augments the Secure Software Development Framework with AI-model-specific practices for model producers, system producers, and acquirers. It emphasizes that AI systems inherit conventional software risks while adding challenges around models, data, and blurred code/data boundaries. It is a community profile for risk-based use, not a certification checklist.

Threats differ by layer

Layer	Representative compromise	Potential consequence
Models and adapters	Tampered file, substituted version, malicious or unsafe adapter	Changed behavior, hidden trigger, unsafe loading, integrity loss
Data	Poisoned, unauthorized, mislabeled, or silently replaced dataset	Backdoors, degraded behavior, privacy or licensing exposure
Code and packages	Dependency confusion, compromised package, vulnerable library	Build or runtime code execution, secret theft, altered pipeline
Build and registry	Stolen publisher identity, mutable tag, compromised builder	Trusted-looking malicious artifact reaches deployment
Runtime and container	Vulnerable base image, excess privilege, exposed management plane	Workload escape, data theft, service compromise
Hosted services	Provider or API change, account compromise, weak data terms	Behavioral drift, availability loss, disclosure, unexpected processing
Prompts and policy	Unauthorized configuration edit or unsafe rollout	Weakened controls, new data access, altered tool behavior
Tools and plugins	Compromised integration or excessive credential	Unauthorized actions and expansion into downstream systems

OWASP's LLM03:2025 guidance explicitly extends supply-chain risk beyond ordinary packages to training data, pretrained models, adapters, and deployment platforms. Risk still depends on architecture: downloading an executable model format into a privileged environment is different from calling a constrained hosted API.

Build an AI artifact ledger

A useful inventory records more than a component name. For each production dependency, capture:

canonical artifact or service identifier;
supplier, source repository, registry, or endpoint;
immutable version, digest, or provider release identifier;
signature or attestation and the identity that produced it;
build process, source revision, builder, and dependency tree where available;
dataset origin, owner, license, transformations, and approvals;
security review and evaluation evidence;
deployment locations, consumers, owner, and retirement date;
known limitations or evidence that could not be verified.

This ledger connects procurement, build, deployment, and incident response. A spreadsheet that cannot link a vulnerable component to running services is documentation, not operational inventory.

Establish model provenance

For downloaded weights, record the publisher identity, exact repository and revision, filename, format, size, cryptographic digest, signature when available, model card, license, evaluation evidence, and approved use. Retrieve artifacts from controlled registries and verify them before loading. Avoid mutable aliases as the only production reference.

A matching hash proves that bytes equal the approved bytes; it does not prove the model is non-malicious, accurate, private, or suitable. A valid signature links an artifact to a signing identity and protects integrity only if the signing key, identity, and verification policy are trustworthy. Sigstore's model-signing work demonstrates practical signing and transparency approaches, but verification must still be paired with supplier review and behavior testing.

Treat conversions, quantization, merges, and adapters as new artifacts. Record inputs, tooling, parameters, builder, outputs, and tests. Combining a trusted base model with an unreviewed adapter does not preserve the base model's trust decision.

Preserve dataset provenance and lineage

For training, fine-tuning, evaluation, and retrieval data, capture source, owner, collection conditions, license or permitted use, consent where relevant, time range, classification, integrity, filtering, labeling, deduplication, transformations, and the model or index versions that consumed it.

Complete lineage may be unavailable for a third-party model. Record the gap, then compensate with supplier controls, constrained deployment, representative evaluations, and monitoring.

NIST AI 100-2 provides a taxonomy for adversarial machine-learning attacks including poisoning and evasion. Taxonomies help structure threats; actual feasibility depends on attacker access, training process, data volume, and model behavior.

Secure packages and builds

Use lockfiles and immutable versions, restrict package sources, review new and transitive dependencies, scan for known vulnerabilities and malicious packages, remove unused components, and define update ownership. Build in isolated workers with short-lived credentials and restricted network access. Protect source branches, release approvals, package publication, signing keys, and registry permissions.

Generate an SBOM for software and container dependencies where it supports vulnerability and license response. Verify provenance before promotion. The SLSA specification defines tracks and increasing guarantees for source and build supply chains, including recommended provenance formats. SLSA evidence strengthens traceability against build tampering; it does not describe dataset quality or model safety.

Harden containers and inference runtimes

Choose maintained, minimal base images; scan operating-system and language packages; remove build tools from runtime images; run without root; use read-only filesystems where practical; minimize Linux capabilities; isolate GPU and management interfaces; restrict egress; and keep secrets outside images and model context.

Manage hosted model and API risk

A hosted model replaces artifact custody with a supplier and service dependency. Document the provider, endpoint, account, region where relevant, model identifier, available version controls, retention and training terms, data-processing commitments, authentication, quotas, fallback behavior, and change-notification process.

Where version pinning exists, use it deliberately; otherwise detect drift with representative evaluations. Minimize submitted data, scope credentials, and avoid silently routing sensitive traffic to an unapproved fallback.

Treat prompts and configuration as supply-chain artifacts

System prompts, retrieval templates, tool schemas, policy files, model settings, and feature flags can materially change authority and behavior. Keep them in version control or another governed configuration store. Require review, record the deploying identity, sign or attest release bundles where useful, and connect every production request to a configuration version.

Keep secrets out of prompts and regression-test configuration changes that alter tools, data sources, approvals, or output handling.

Govern tools, plugins, and external services

Every integration introduces supplier code, schemas, credentials, data flows, and operational dependencies. Record the owner, package or endpoint, permissions, destinations, data classes, version, review evidence, and revocation path. Prefer narrow typed tools, allowlisted destinations, and credentials issued after deterministic authorization. The AI agent permissions guide explains why the model cannot be the authorization engine.

The LLM security risks guide covers runtime trust boundaries; supply-chain approval never grants a tool unlimited authority.

SBOMs and emerging AI BOMs

An SBOM inventories software components and supports vulnerability, license, and incident workflows. AI systems also need evidence about models, adapters, datasets, prompts, evaluation sets, and hosted services. OWASP describes AI and ML bills of materials as an emerging area. Avoid claiming one format currently captures every AI-specific relationship.

Optimize for answerable operational questions: Which model digest is live? Which dataset produced it? Which deployments include an affected package? Preserve relationships between records rather than forcing all evidence into one flat document.

The AI Supply Chain Control Matrix

PermsAI's matrix connects prevention to response across the full system.

Component	Threat	Provenance evidence	Preventive control	Detection and response evidence
Model or adapter	Substitution, tampering, unsafe source	Publisher, revision, digest, signature, model card	Approved registry, verification gate, safe format, isolated evaluation	Load digest, behavior tests, deployment inventory, revocation
Dataset or embeddings	Poisoning, unauthorized use, lineage loss	Source, owner, license, version, transformations	Source approval, access control, validation, separated evaluation data	Ingestion logs, anomaly review, consuming models/indexes
Code and packages	Malicious or vulnerable dependency	Repository, lockfile, SBOM, source/build provenance	Restricted registries, review, isolated reproducible build	Scanner results, advisory match, affected deployment query
Container and runtime	Vulnerable base or privileged configuration	Image digest, base image, runtime and deployment manifest	Minimal image, non-root, patching, isolation, secret controls	Runtime telemetry, configuration drift, workload inventory
Prompt and policy bundle	Unauthorized or unsafe change	Source revision, reviewer, bundle digest, release record	Protected review, tests, separation of duties	Request-to-config correlation, rollback history
Hosted model or API	Drift, compromise, outage, data-policy mismatch	Supplier, endpoint, version, terms, review date	Vendor assessment, data minimization, scoped keys, controlled fallback	Change alerts, eval trends, usage logs, failover decision
Tool or plugin	Compromised integration or excessive access	Supplier, schema, permissions, destination, version	Allowlist, typed gateway, least privilege, server-side authorization	Tool calls, policy decisions, results, credential revocation

Use verify, quarantine, release, and revoke gates

A defensible artifact path has four states:

Verify: authenticate the source; check digest, signature, provenance, license, policy, and expected type.
Quarantine: scan, inspect, and evaluate untrusted models, data, containers, and packages in an isolated environment.
Release: promote only immutable approved artifacts through a controlled pipeline; record approver and deployment targets.
Revoke: block a compromised digest, supplier, key, dataset, configuration, or service and identify every dependent deployment.

Red-team and functional evaluations belong inside release decisions, particularly for third-party models and major data changes. The AI red teaming guide shows how to turn a supply-chain scenario into repeatable evidence. The RAG security guide applies source, provenance, authorization, and deletion controls to retrieval systems.

Prepare updates and incident response

Monitor advisories, supplier changes, repository ownership, expiring signatures, model deprecations, dataset corrections, runtime vulnerabilities, and unexpected behavioral drift. Stage updates, rerun security and workload evaluations, and use gradual rollout with rollback criteria.

During an incident, responders should trace component to deployment, preserve provenance and deployment evidence, quarantine affected artifacts, revoke trust, and verify replacements before restoration.

Practical checklist

Inventory models, adapters, datasets, embeddings, packages, containers, prompts, tools, and services.
Assign an owner and canonical identifier to every production dependency.
Prefer immutable versions and verify hashes, signatures, and provenance where available.
Preserve dataset ownership, license, classification, transformation, and consumer lineage.
Restrict repositories, registries, builders, signing identities, and deployment approvals.
Generate and retain SBOMs for software and container components.
Build in isolated workers with minimal, short-lived credentials.
Quarantine and evaluate third-party models, adapters, conversions, and data.
Run minimal non-root workloads and keep secrets outside images, prompts, and logs.
Document hosted-provider versioning, data handling, change, outage, and fallback behavior.
Govern prompts, policy files, schemas, and feature flags as versioned release artifacts.
Connect every component record to deployed services and a tested revocation path.
Monitor advisories and drift; stage, evaluate, and roll back updates safely.
Record evidence gaps and residual risk instead of overstating assurance.

Related artifact and poisoning evidence

Use Data and Model Poisoning for detection and mitigation across lifecycle stages, and Open-Source Model Security Research for the current evidence on open-weight artifacts, repositories, loaders, and serving boundaries.