As many enterprises move to running AI training or inference on their data, the data and the code need to be protected, especially for large language models…
Overview
NVIDIA has announced the general availability of Secure AI, focusing on protecting data and code during AI training and inference, particularly for large language models (LLMs). The article details the features of Protected PCIe (PPCIE) mode on NVIDIA HGX H100 and H200 systems, emphasizing the importance of security in handling sensitive data.
What You'll Learn
1
How to implement Protected PCIe mode on NVIDIA GPUs
2
Why security is essential for AI training and inference
3
When to use NVIDIA Confidential Computing for sensitive data
Prerequisites & Requirements
- Understanding of AI training and inference processes
- Familiarity with NVIDIA CUDA and driver installation(optional)
- Experience with GPU-based computing environments
Key Questions Answered
What is Protected PCIe mode and how does it work?
Protected PCIe mode (PPCIE) is a security feature that protects multi-GPU configurations in a Confidential Virtual Machine. It safeguards data in use by ensuring that the configuration of GPUs and switches is secure, preventing unauthorized access or modifications.
What hardware is required for implementing PPCIE?
To implement Protected PCIe mode, a system must have NVIDIA H100 or H200 Tensor Core GPUs on an HGX 8-GPU system, along with a CPU that supports a trusted execution environment (TEE) such as AMD SEV-SNP or Intel TDX.
What software is needed to get started with Secure AI?
The software requirements for Secure AI include the NVIDIA driver (CUDA 12.8 or later), NVIDIA firmware 1.7.0 or later, and compatible hypervisors like Microsoft Azure Hyper-V or KVM. Operating systems supported include Ubuntu 25.04 for AMD and Ubuntu 24.04 with patches for Intel.
Technologies & Tools
Hardware
Nvidia H100 Tensor Core
Used for implementing Protected PCIe mode in Secure AI.
Hardware
Nvidia H200 Tensor Core
Also used for implementing Protected PCIe mode in Secure AI.
Software
Cuda
Required driver for running Secure AI applications.
Key Actionable Insights
1Leverage Protected PCIe mode to enhance the security of your AI models during training and inference.This is particularly important for enterprises handling sensitive data, as it ensures that data remains secure even while in use, mitigating risks associated with data breaches.
2Consider using NVIDIA Confidential Computing to avoid performance trade-offs when securing AI workloads.This technology allows enterprises to maintain high performance while ensuring that their data and models are protected from unauthorized access.
3Stay updated with the latest NVIDIA firmware and driver versions to ensure compatibility and security.Using the latest versions helps in leveraging new security features and optimizations that NVIDIA continuously rolls out.
Common Pitfalls
1
Failing to secure data in use can lead to vulnerabilities during AI model training and inference.
Many organizations focus only on data-at-rest and data-in-motion security, neglecting the critical need to secure data while it is being processed, which can expose sensitive information.
Related Concepts
Nvidia Confidential Computing
Data Security In AI Applications
Trusted Execution Environments