The introduction of the NVIDIA Jetson Orin Nano Super Developer Kit sparked a new age of generative AI for small edge devices. The new Super Mode delivered an…
Overview
NVIDIA JetPack 6.2 introduces Super Mode for the Jetson Orin Nano and Jetson Orin NX modules, significantly enhancing generative AI performance. This release enables up to 2x higher performance for AI models and introduces new power modes for optimized energy consumption.
What You'll Learn
1
How to enable Super Mode on Jetson Orin modules using JetPack 6.2
2
Why Super Mode can double the performance of generative AI models
3
How to utilize the updated Power Estimator Tool for custom power profiles
Prerequisites & Requirements
- Understanding of NVIDIA Jetson modules and AI model performance
- Familiarity with JetPack SDK and flashing configurations(optional)
Key Questions Answered
What performance improvements does JetPack 6.2 offer for Jetson Orin modules?
JetPack 6.2 provides up to 2x higher generative AI model performance on Jetson Orin Nano and Jetson Orin NX modules. This enhancement is particularly beneficial for robotics and edge AI applications, allowing developers to unlock greater value and lower total cost of ownership.
How can developers enable the new power modes in JetPack 6.2?
To enable the new power modes, developers must use the new flashing configuration 'jetson-orin-nano-devkit-super.conf' while flashing the Jetson Orin modules. After flashing, the command 'sudo nvpmodel -m 2' for Nano or 'sudo nvpmodel -m 0' for NX can be used to activate MAXN SUPER mode.
What benchmarks were conducted for large language models using Super Mode?
Benchmarks showed significant performance gains for large language models on Jetson Orin modules. For example, the performance for Gemma 2 2B improved from 11.40 tokens/sec to 18.60 tokens/sec on Orin Nano 8GB, achieving a performance gain of 1.64 times.
Key Statistics & Figures
Generative AI performance boost
up to 2x
This applies to generative AI models on Jetson Orin Nano and Jetson Orin NX modules.
Performance gain for Gemma 2 2B
1.64x
Measured on Jetson Orin Nano 8GB with Super Mode.
Technologies & Tools
Software
Nvidia Jetpack
Used to enhance performance and enable new features for Jetson modules.
Hardware
Jetson Orin Nano
Target platform for the performance improvements and new power modes.
Hardware
Jetson Orin Nx
Target platform for the performance improvements and new power modes.
Key Actionable Insights
1Developers should leverage the new Super Mode to enhance the performance of their AI applications significantly. By enabling this mode, they can achieve up to 2x higher inference performance, which is crucial for demanding applications.This is particularly important for applications in robotics and edge AI, where performance directly impacts operational efficiency and capabilities.
2Utilizing the updated Power Estimator Tool can help developers create custom power profiles that balance performance and power consumption effectively.This is essential for optimizing the energy efficiency of applications, especially in battery-powered or energy-sensitive environments.
Common Pitfalls
1
Failing to update the flashing configuration can prevent access to new power modes.
Developers must ensure they use the 'jetson-orin-nano-devkit-super.conf' configuration when flashing to enable the new features. Not doing so will result in the inability to utilize the performance enhancements.
Related Concepts
Generative AI
Power Management In Embedded Systems
Performance Benchmarking For AI Models