NVIDIA logo

How NVIDIA Uses PyTorch

631 engineering articles about PyTorch from NVIDIA's engineering team

Articles

Filter:
NVIDIA logo
NVIDIA
Advanced
AI applications at the edge often need to run multiple neural networks within limited compute and power budgets. NVIDIA Deep Learning Accelerator (DLA)…
Nikshay Shrivastava
10 min read
Includes Code
--
NVIDIA logo
NVIDIA
Intermediate
Autonomous vehicle (AV) developers need reasoning models that can operate within the compute and latency constraints of an onboard system.
Marco Pavone
12 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
During reinforcement learning (RL) post-training, training trajectories are generated on the fly by running inference with the current policy model.
Tanya Lenz
11 min read
Includes Code
--
NVIDIA logo
NVIDIA
Intermediate
As distributed AI training scales, failures across hardware and software dependencies increasingly disrupt training progress, making fast…
Elizabeth Goodman
12 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
Physical AI developers working with world models face two distinct challenges: adapting models to their sensors, tasks, and data, and getting those models into…
Elizabeth Goodman
9 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
Atomistic simulations help researchers understand how molecules and materials behave, from chemical reactions and phase transitions to mechanical failure.
Anoushka Bhutani
11 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
NVIDIA CUDA Tile (cuTile) is a Python-based DSL and parallel programming model for NVIDIA GPUs that automatically leverages advanced hardware capabilities to…
Tanya Lenz
14 min read
Includes Code
--
NVIDIA logo
NVIDIA
Beginner
Bring coding agents, model customization, and CUDA-accelerated development to your everyday PC…
Elizabeth Goodman
7 min read
--
NVIDIA logo
NVIDIA
Advanced
Meta launches Muse Spark Open, a coding-optimized 816B mixture-of-experts (MoE) model with 42B active parameters and a 1M token context window.
Tanya Lenz
5 min read
--
NVIDIA logo
NVIDIA
Advanced
Observatories and telescopes, lasers and X-ray light sources, and other high-throughput instruments generate image data faster than CPU-bound pipelines can…
Quynh L. Nguyen
17 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
Bitwise determinism makes large-scale pretraining easier to debug, validate, and resume reproducibly. These benefits become especially valuable when training…
Ashwath Aithal
10 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
Automatic speech recognition must handle how people actually speak, not only the languages and styles that dominate pretraining data.
Elizabeth Goodman
13 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
Generative recommender (GR) systems are emerging as a powerful new approach for large-scale personalization. Instead of treating recommendation as a set of…
Tanya Lenz
10 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
As language models grow, scaling dense architectures becomes increasingly expensive. In a dense transformer, every token passes through every layer…
Michelle Horton
6 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
As large language model (LLM) inference increasingly processes sensitive information and proprietary model context across personal, enterprise…
Tanya Lenz
6 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
The compute and memory demands of generative AI increasingly exceed what a single GPU can provide. NVIDIA TensorRT multi-device inference is a new capability…
Tanya Lenz
6 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
Weather-sensitive industries increasingly have access to observations that offer an earlier, more local view of changing conditions.
Elizabeth Goodman
13 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
cuTile Rust () is a tile-based system for safe, idiomatic GPU kernel authoring in the Rust programming language. Extending the Rust ownership model to tile…
Tanya Lenz
18 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
Biomolecular structure prediction is now often run at proteome scale, where the goal is to move an entire worklist through the pipeline efficiently.
Elizabeth Goodman
10 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
Every NVIDIA CUDA Toolkit release adds functionality and performance improvements that help developers get more from NVIDIA GPUs and the broader NVIDIA software…
Jonathan Bentz
14 min read
Includes Code
--
NVIDIA logo
NVIDIA
Intermediate
Open AI models are evolving faster than ever, but bringing them into native applications can still require model-specific conversion, preprocessing…
Tanya Lenz
6 min read
Includes Code
--
NVIDIA logo
NVIDIA
Intermediate
Alibaba released the model weights for Qwen3.8-Flash-Next as a preview of the upcoming Qwen4 architecture for developers to experiment with and evaluate.
Michelle Horton
3 min read
--
NVIDIA logo
NVIDIA
Advanced
For years, a Python developer who needed a GPU had two realistic choices: Learn NVIDIA CUDA C++ well enough to write an extension, set up a build toolchain…
Elizabeth Goodman
12 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
Use AdaptGrow, a GPU-accelerated matrix factorization algorithm, to turn rolling correlation and tail-dependence matrices into hard clusters…
Elizabeth Goodman
12 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
Recommender systems (RecSys) are one of the most ubiquitous machine learning problems in the consumer internet industry yet notoriously difficult to train and…
Elizabeth Goodman
11 min read
Includes Code
--
NVIDIA logo
NVIDIA
Intermediate
NVIDIA Holoscan is a platform for building real-time AI applications at the edge, from medical imaging to robotics. HoloHub is its companion repository: a…
Elizabeth Goodman
10 min read
Includes Code
--
NVIDIA logo
NVIDIA
Intermediate
Modern vision-language models (VLMs) can support tasks such as visual question answering, captioning, and image-text reasoning. In practice, however…
NVIDIA logo
NVIDIA
Intermediate
Atomistic simulation requires three things: knowledge of the science, compute-efficient implementation of simulations, and accessible interfaces to the…
Elizabeth Goodman
11 min read
Includes Code
--
NVIDIA logo
NVIDIA
Intermediate
Teams customize their models to hit their targets for latency, speed, memory, and compute. With the open NVIDIA Nemotron family of models…
Tanya Lenz
18 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
Alibaba released the open weights for Qwen3.8-2.4T-A95B (Qwen3.8-Max), its largest open-weight model, bringing near-frontier capabilities to the open ecosystem.
Michelle Horton
4 min read
--
NVIDIA logo
NVIDIA
Advanced
NVIDIA nvmath-python is a library designed to bridge the gap between the Python scientific community and NVIDIA CUDA-X math libraries. It gives Python users…
Michelle Horton
14 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
Frontier model pre-training has converged on mixture of experts (MoE), which is fundamentally changing what limits large-scale AI training. As compute per token…
Kirthi Devleker
7 min read
--
NVIDIA logo
NVIDIA
Intermediate
Developers building 3D, design, simulation, robotics, and industrial digital twin applications need ways to bring physical AI capabilities into the tools and…
Tanya Lenz
16 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
Useful quantum computers will require fault tolerant logical operations. Researchers are actively exploring many different quantum error correction (QEC) codes…
Elizabeth Goodman
5 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
There are many ways to optimize code for GPUs. In this post, you’ll learn how kernel fusion can improve memory bandwidth and reduce kernel launch overhead…
Michelle Horton
10 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
Biomolecular structure prediction and co-folding with models like OpenFold3 are now mainstream, large-scale workloads powering drug discovery and protein design.
Elizabeth Goodman
8 min read
Includes Code
--
NVIDIA logo
NVIDIA
Intermediate
NVIDIA Omniverse NuRec is a neural reconstruction pipeline for building high-fidelity 3D representations of real-world environments from multisensor data such…
Tanya Lenz
8 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
Generative AI workloads are rapidly outgrowing the memory and compute budget of single GPUs. For inference developers building media generation pipelines…
Peter Kisfaludi
10 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
As AI systems move from single-turn interactions to coordinated multiagent workflows, low-latency inference becomes increasingly important.
Amr Elmeleegy
7 min read
--
NVIDIA logo
NVIDIA
Advanced
Foundation models are reshaping computational biology. Pretrained on massive corpora of protein or genomic sequences, models such as ESM2 (a protein language…
Bruno Alvisio
11 min read
Includes Code
--
NVIDIA logo
NVIDIA
Intermediate
As enterprise AI adoption scales, developers are increasingly forced to stitch together fragmented pipelines—separate models for text, vision…
Anu Srivastava
5 min read
Includes Code
--
NVIDIA logo
NVIDIA
Intermediate
This post is the third of a three-part series. See also Model Quantization: Concepts, Methods, and Why It Matters and Model Quantization: Post-Training…
Ruixiang Wang
9 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
AI agents are changing how you interact with your PC. Creators, developers, and AI enthusiasts are already using these agents extensively to assist with day-to…
Annamalai Chockalingam
8 min read
--
NVIDIA logo
NVIDIA
Advanced
Each wave of AI has created a new scaling law. Pretraining scaled intelligence through larger datasets, more parameters, and massively parallel GPU systems.
Praveen Menon
7 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
AI applications are moving beyond text generation to multimodal systems that can perceive, search, and reason across images, documents, video…
Anu Srivastava
4 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
In production inference deployments, demand fluctuates over time, requiring inference replicas to scale elastically. However, cold-starting inference workloads…
Schwinn Saereesitthipitak
14 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
Large language models (LLMs) are revolutionizing the financial trading landscape by enabling sophisticated analysis of vast amounts of unstructured data to…
Dan Blanaru
10 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
NVIDIA CUDA 13.3 brings new capabilities and performance optimizations to developers across the CUDA ecosystem. The launch of NVIDIA CUDA Tile programming in…
Jonathan Bentz
13 min read
Includes Code
--
NVIDIA logo
NVIDIA
Advanced
The path from a trained AI model to production should be smooth, but rarely is. Many teams invest weeks fine-tuning models, only to discover that exporting to a…
Lovina Dmello
10 min read
Includes Code
--
NVIDIA logo
NVIDIA
Intermediate
This post is the second of a three-part series. See also Model Quantization: Concepts, Methods, and Why It Matters and Model Quantization: Turn FP8 Checkpoints…
Ruixiang Wang
8 min read
Includes Code
--