Understanding Memory Management on Hardware-Coherent Platforms

If you’re an application developer or a cluster administrator, you’ve likely seen how non-uniform memory access (NUMA) can impact system performance.

NVIDIA logo
NVIDIA
•
Kumar Sankaran
•6 min read•advanced•
--
•View Original
Kubernetes

Overview

The article discusses memory management on hardware-coherent platforms, specifically focusing on the differences between Non-Uniform Memory Access (NUMA) and Coherent Driver-based Memory Management (CDMM) modes. It highlights how CDMM can enhance application performance by allowing the NVIDIA driver to manage GPU memory directly, especially in environments like Kubernetes.

What You'll Learn

1

How to implement Coherent Driver-based Memory Management (CDMM) mode for optimal GPU memory control

2

Why Non-Uniform Memory Access (NUMA) can lead to performance issues in GPU applications

3

When to use CDMM mode over NUMA for Kubernetes deployments

Prerequisites & Requirements

  • Understanding of memory management concepts, particularly NUMA and GPU memory
  • Familiarity with NVIDIA drivers and CUDA APIs(optional)

Key Questions Answered

What is the difference between NUMA and CDMM modes?
NUMA mode allows the OS to manage both CPU and GPU memory, treating GPU memory as a generic pool. In contrast, CDMM mode separates GPU memory management from the OS, allowing the NVIDIA driver to control GPU memory directly, which can lead to better performance.
How does CDMM mode affect Kubernetes performance?
In CDMM mode, GPU memory is not exposed to the OS, preventing issues like memory over-reporting and incorrect pod memory limits that can occur in NUMA mode. This leads to improved performance and isolation for Kubernetes pods.
What are the unintended consequences of hardware-coherent platforms?
Hardware-coherent platforms may allow the OS to use GPU memory unexpectedly, such as for caching or avoiding out-of-memory conditions. This can disrupt applications optimized for specific memory layouts, making CDMM mode a preferable choice.
What tools should not be used in CDMM mode?
Tools like numactl and mbind should not be used for GPU memory management in CDMM mode, as they will have no effect on GPU memory. They can still be utilized for managing system memory.

Technologies & Tools

Some links below are affiliate links. We may earn a commission if you make a purchase.

Software
Nvidia Driver
Manages GPU memory in CDMM mode
Software
Cuda
APIs used for memory allocation on both CPU and GPU
Orchestration
Kubernetes
Platform where CDMM mode can enhance performance

Key Actionable Insights

1
Consider switching to CDMM mode if your applications rely heavily on GPU memory management, especially in Kubernetes environments.
This mode allows for better control over GPU memory, which can prevent performance issues related to memory over-reporting and pod isolation.
2
Evaluate your application's memory management strategy to determine if NUMA mode is causing performance inconsistencies.
Understanding how NUMA impacts memory allocation can help you optimize your application for better performance on hardware-coherent platforms.
3
Utilize the NVIDIA whitepaper on CDMM for detailed implementation guidance.
This resource provides comprehensive insights into enabling CDMM mode and maximizing the performance of your GPU-accelerated workloads.

Common Pitfalls

1
Using NUMA mode without understanding its implications can lead to performance issues, such as memory over-reporting in Kubernetes.
This happens because Kubernetes may incorrectly account for GPU memory, leading to resource allocation failures. Switching to CDMM mode can mitigate these issues.

Related Concepts

Memory Management
Non-uniform Memory Access (numa)
Coherent Driver-based Memory Management (cdmm)
Kubernetes Performance Optimization