There's a question I ask when I'm evaluating the security architecture of an AI deployment, and it's one that usually produces a long pause: "What is the root of trust for your model weights?"
The typical response describes encryption: weights are encrypted at rest, encrypted in transit, access-controlled through IAM policies, audited through CloudTrail or equivalent. All of that is correct, and all of it is insufficient.
Encryption protects data from unauthorized parties who don't control the hardware. It provides no protection against an adversary who does — whether that adversary is a compromised hypervisor, a malicious co-tenant, a data center insider, or a nation-state actor with the access and motivation to compromise physical infrastructure. The keys that protect your encrypted model weights are ultimately decrypted by software running on hardware. If you can't trust the hardware, you can't trust the software, and you can't trust the keys.
This is the problem that hardware root-of-trust is designed to solve. It's a problem that the cryptographic hardware security community has been working on for decades, and the solutions — TPMs, secure enclaves, attestation protocols, confidential computing — are mature enough to deploy in production cloud AI infrastructure today. Most AI teams are not deploying them, for reasons that have more to do with awareness gaps than technical barriers.
What "Root of Trust" Actually Means
The phrase "root of trust" has a specific technical meaning that gets lost in casual usage. Let me be precise.
A root of trust is a component whose trustworthiness is assumed rather than derived. Every security system has one. The question is whether it's implicit and unexamined, or explicit and hardware-anchored.
In a conventional software security model, the root of trust is implicitly the operating system kernel or, more precisely, the hypervisor in a virtualized environment. If the hypervisor is not compromised, the security properties of VMs running on it hold. If the hypervisor is compromised — through a vulnerability, a malicious firmware update, an insider action, or a supply chain attack — all security properties of all workloads running on that hardware are void, regardless of how strong the software-layer security is.
A hardware root of trust relocates the trust anchor from software to a dedicated hardware component designed with specific security properties:
Tamper resistance. The hardware is designed to destroy sensitive material (keys, attestation data, measurement values) if physical tampering is detected. The Trusted Platform Module (TPM) specification requires zeroization of all cryptographic keys upon tamper detection.
Immutable measurement. During boot, each component of the software stack is measured — a cryptographic hash of the code is computed and recorded — before it executes. The measurements are stored in Platform Configuration Registers (PCRs) inside the TPM, where they cannot be modified after the fact by software running on the system. An attestation query to the TPM returns a signed record of these measurements, allowing a verifying party to confirm that a specific, unmodified software stack is running.
Key isolation. Cryptographic keys stored in a TPM or hardware security module (HSM) never leave the hardware in cleartext. Decryption operations are performed inside the hardware boundary. An adversary who compromises the OS or hypervisor layer cannot extract the keys; they can at most ask the hardware to decrypt specific ciphertext, and the hardware can be configured to refuse decryption unless the current measured state of the system matches an expected value.
Attestation. The combination of immutable measurement and key isolation enables remote attestation: a protocol where the hardware can prove to a remote party that it is running specific, unmodified software in a specific, unmodified configuration. Attestation is what allows a cloud service provider to tell a customer: "The AI inference workload you just submitted ran on hardware with this specific firmware, this specific hypervisor, and this specific runtime. Here is a cryptographically verifiable proof of that statement."
These capabilities exist in deployed hardware today. TPMs are standard in enterprise laptops. HSMs are standard in payment processing infrastructure. The question is whether they're being used for AI infrastructure — and for the most sensitive AI workloads, the answer needs to be yes.
The AI-Specific Threat Landscape
The hardware security threats that motivate root-of-trust architecture are not hypothetical for AI infrastructure. Let me walk through the threat categories that are specific to or amplified by AI workloads.
Model weight confidentiality in multi-tenant inference. A model trained on proprietary data with significant commercial value represents an asset that a competitor or adversary would pay substantial money to acquire. In a multi-tenant cloud inference deployment, the model weights are loaded into GPU memory by a software stack that includes the cloud provider's hypervisor, the GPU driver, and the inference runtime. A compromise at any layer of that stack could expose the weights.
I've written separately about side-channel attacks on ML accelerators as a mechanism for weight extraction. Hardware root-of-trust addresses a different but related threat: direct memory access to model weights by a compromised software layer. Confidential computing protections — specifically, encrypted GPU memory with hardware-enforced access control — address this by ensuring that model weights are decryptable only within an attested, verified execution environment.
Training pipeline integrity. A model is only as trustworthy as the process that trained it. Adversarial training data poisoning attacks — where malicious data is injected into the training set to implant backdoors or degrade performance — are a known threat. But there's a hardware-layer analog: an attacker with access to the training infrastructure can potentially modify gradients, alter checkpoints, or inject malicious initialization values in ways that would be invisible to software-layer monitoring.
The hardware trojan research I've discussed elsewhere is relevant here: a hardware trojan in a training accelerator could, in principle, apply targeted modifications to specific weight values during training that would be nearly undetectable through software-layer validation. This is a high-sophistication attack that requires supply chain access, but for nation-state adversaries or sophisticated criminals targeting AI systems in critical infrastructure, "high sophistication" is not the same as "impossible."
Inference result integrity. In applications where AI inference outputs are used to make consequential decisions — medical diagnosis, fraud detection, autonomous vehicle perception — the integrity of those outputs matters in a specific way: you need to be able to verify that the output was produced by the intended model running correctly, not by a compromised model or a software layer that modified the output after the model produced it.
Attestation-backed inference provides this property: a signed proof that a specific model version, running in a specific verified environment, produced a specific output. This is the AI equivalent of the signed code execution that code integrity protections provide — and it has the same value proposition: it converts a trust assumption ("I believe this was produced correctly") into a verifiable claim ("here is cryptographic evidence that it was").
Regulatory and audit requirements. This is increasingly a practical driver rather than a theoretical one. Healthcare AI deployments subject to HIPAA have audit requirements that implicitly require some form of hardware-level assurance that data wasn't exposed to unauthorized parties. EU AI Act provisions around high-risk AI systems are moving toward requirements that will need to be supported by hardware-level attestation. Financial services regulators are beginning to ask about the hardware-level security of AI systems used in credit decisions and fraud detection. Organizations that have not built hardware root-of-trust into their AI infrastructure are building a compliance debt that they will have to repay eventually — ideally before a regulator asks the question.
The Technology Stack for Hardware-Anchored AI Security
The good news for AI teams evaluating hardware root-of-trust: the building blocks exist, are production-ready, and are increasingly supported by major cloud providers. The bad news: they require deliberate architecture decisions and are not the default configuration for any standard AI deployment.
Confidential Computing. This is the primary technology category for AI workload protection. Confidential computing creates a hardware-isolated execution environment — called a Trusted Execution Environment (TEE) — where code and data are protected from the host OS, hypervisor, and other VMs by hardware memory encryption and access controls enforced by the CPU.
Intel TDX (Trust Domain Extensions) and AMD SEV-SNP (Secure Encrypted Virtualization - Secure Nested Paging) are the two primary implementations for server-class hardware. Both encrypt VM memory at the hardware level with keys that are managed by the CPU and never accessible to the hypervisor. Both support attestation protocols that allow a workload to prove to a remote party that it's running in a genuine TEE with specific, verified properties.
For AI inference specifically, the critical gap has been GPU support. TEEs protect CPU-executed code and memory, but model inference runs primarily on GPUs. NVIDIA has addressed this with the H100's Confidential Computing support, which extends hardware memory encryption to GPU memory and provides attestation for GPU workloads. This is the capability that makes confidential computing practically useful for AI inference — and it's available today in production cloud deployments at AWS, Azure, and GCP.
Hardware Security Modules (HSMs). For key management infrastructure supporting AI deployments — specifically, the keys that encrypt model weights in storage and control access to training data — HSMs provide the hardware root-of-trust that software-based key management systems cannot. The distinction is meaningful: a software KMS stores keys in memory on a system that can be compromised by a hypervisor-level attacker. An HSM stores keys in tamper-resistant hardware that will destroy them before permitting extraction.
Cloud HSMs (AWS CloudHSM, Azure Dedicated HSM, GCP Cloud HSM) provide this capability without requiring on-premises hardware. For AI deployments with regulatory requirements or high-value IP protection needs, the incremental cost of HSM-backed key management versus software KMS is small relative to the security difference.
Secure Boot and Measured Boot. For AI infrastructure that runs on hardware you control — on-premises GPU clusters, edge AI deployments — secure boot and measured boot establish a hardware-anchored chain of trust from firmware through bootloader through OS through inference runtime. Each component is measured before execution, and the measurement chain is rooted in the TPM. This allows you to verify that the software stack running your AI workload is exactly what you intended — no modifications, no injected code, no unauthorized components.
This is standard practice in security-sensitive enterprise deployments. It is not standard practice in AI infrastructure deployments, where the emphasis has historically been on compute performance and availability rather than security.
Attestation Services. Attestation is the runtime mechanism that makes hardware root-of-trust useful in a distributed system. An attestation service verifies TEE measurements against expected values and issues attestation tokens that other services can validate. The Intel Attestation Service, AMD's attestation infrastructure, and NVIDIA's GPU attestation service are the primary examples. Building attestation into your AI inference pipeline — so that clients can verify they're interacting with an attested model running in a verified environment — is the mechanism that converts hardware security guarantees into application-level assurance.
What This Looks Like in Practice
Let me describe what a hardware root-of-trust architecture for a production AI inference deployment actually looks like, based on the patterns I've seen work in security-conscious deployments.
Model weight protection: Model weights are encrypted with a key that is wrapped (encrypted) by an HSM-managed key. The unwrapped key is only available inside a confidential compute instance. Note: hardware root-of-trust addresses the software-layer and hypervisor threat, but side-channel attacks on ML accelerators — which exploit power, electromagnetic, and timing emissions from the hardware itself — require additional physical-layer defenses beyond what confidential computing provides. The confidential compute instance boots with a measured software stack, and the key unwrapping is conditional on a successful attestation: the HSM releases the key only if the current instance presents a valid attestation showing it's running the expected inference runtime in an unmodified TEE. A compromised hypervisor cannot access the weights because it can never obtain the decryption key.
Inference result signing: Each inference result is signed by a key held inside the TEE. The signature is over the model version identifier, the input hash, and the output. A downstream service that consumes the inference result can verify the signature against a public key published in an attestation certificate, confirming that the result was produced by the expected model in the expected environment without modification.
Audit trail integrity: Audit logs from the inference service are written to an append-only log that is also inside the TEE boundary, signed with a TEE-held key, and replicated to tamper-evident log storage. This provides the audit trail integrity that regulators require without relying on the trustworthiness of the underlying OS or hypervisor to protect log integrity.
The complexity of this architecture is real. It requires expertise in confidential computing, HSM management, and attestation protocol design that most AI engineering teams don't have in-house. It's worth building that expertise or engaging specialists — not for every AI deployment, but for the deployments where the stakes warrant it.
The Gap Between AI Security and Hardware Security
The AI security discipline, as it has developed over the past five years, has focused heavily on adversarial examples, data poisoning, model extraction via API queries, and prompt injection in LLM systems. These are real threats and the research addressing them is valuable.
But this focus has left a gap: the hardware layer beneath the AI stack has been treated as out-of-scope for AI security, implicitly trusted, and essentially ignored by AI security teams. The hardware security community — where I spent my PhD years and where I continue to track the research — has been addressing these threats for decades, with mature solutions that are now available in production cloud infrastructure.
The integration of these two communities — AI security and hardware security — is happening, but slowly. The organizations that build the integration first will have a security architecture that addresses the full threat landscape. The organizations that wait for the integration to become industry standard will have a security architecture that addresses most threats and has a known, exploitable gap at the hardware layer.
On the supply chain side, my analysis of FPGA security in critical infrastructure extends these themes to the reconfigurable hardware layer that's increasingly common in cloud and edge AI deployments — FPGAs and SmartNICs handling AI inference workloads face both the supply chain threats I've described and additional attack surfaces around bitstream integrity and remote update infrastructure.
It's worth noting that hardware root-of-trust provides strong protection against software-layer attacks and supply chain compromise at the firmware/software level, but has limited effectiveness against physical side-channel attacks operating below the software stack. For the specific threat of differential power analysis (DPA) on neural network accelerators — where an attacker measures power consumption during inference to extract model weights — see my analysis of DPA on neural networks, which covers circuit-level defenses and why TEEs are necessary but not sufficient for high-threat environments.
My research and the broader hardware security literature have made the threat clear. The ventures I've built have given me direct experience with what it takes to bridge hardware security into AI infrastructure in production. The path is available. What's required now is for AI teams to understand that the hardware layer is their problem too — not a problem that the cloud provider has solved for them, and not a problem that can be ignored because it's someone else's domain.
The silicon that your AI workloads run on is not a neutral substrate. It has physical properties that can be exploited. It was designed and manufactured by humans in a supply chain that can be compromised. It runs firmware that can be modified. Building a security architecture that treats this reality seriously — rather than abstracting it away and hoping the abstraction holds — is the work that the next generation of AI infrastructure security requires.
The hardware root-of-trust is where that work starts. It's past time for AI infrastructure to take it seriously.