NVIDIA Accelerates Apache Spark, World’s Leading Data Analytics Platform

NVIDIA today announced that it is collaborating with the open-source community to bring end-to-end GPU acceleration to Apache Spark 3.0.

NVIDIA logo
NVIDIA
•
Nefi Alarcon
•2 min read•beginner•
--
•View Original
ApacheApache SparkSQL

Overview

NVIDIA is collaborating with the open-source community to introduce end-to-end GPU acceleration to Apache Spark 3.0, enhancing data processing capabilities for over 500,000 data scientists. This advancement allows for integrated AI model training on the same Spark cluster, significantly improving performance and cost efficiency.

What You'll Learn

1

How to apply GPU acceleration to ETL workloads in Apache Spark 3.0

2

Why integrating AI model training with data processing in Spark enhances performance

3

When to leverage GPU-accelerated data analytics for cost savings

Key Questions Answered

How does NVIDIA's GPU acceleration improve Apache Spark 3.0?
NVIDIA's GPU acceleration in Apache Spark 3.0 enhances ETL data processing workloads, allowing data scientists to process AI model training on the same Spark cluster. This integration leads to high-performance analytics across the data science pipeline, significantly improving efficiency without requiring changes to existing Spark code.
What performance improvements has Adobe achieved using Spark 3.0?
Adobe, using a preview release of Spark 3.0 on Databricks, achieved a 7x performance improvement and 90 percent cost savings in initial tests. This was accomplished through GPU-accelerated data analytics for product development within Adobe Experience Cloud.
What industries benefit from the collaboration between NVIDIA and Databricks?
The collaboration between NVIDIA and Databricks benefits various industries, including healthcare, finance, and retail. By optimizing Spark with the RAPIDS software suite, GPU acceleration is brought to data science and machine learning workloads across these sectors.

Key Statistics & Figures

Performance improvement
7x
Achieved by Adobe using Spark 3.0 on Databricks
Cost savings
90 percent
Realized by Adobe in initial tests with GPU-accelerated data analytics

Technologies & Tools

Some links below are affiliate links. We may earn a commission if you make a purchase.

Data Analytics Platform
Apache Spark
Used for big data processing and analytics
Software Suite
Rapids
Optimizes Spark for GPU acceleration in data science and machine learning workloads
Cloud Platform
Databricks
Hosts Spark 3.0 and facilitates GPU-accelerated data analytics

Key Actionable Insights

1
Data scientists should consider adopting Apache Spark 3.0 with GPU acceleration to enhance their ETL processes.
This adoption can lead to significant performance improvements and cost savings, especially for large datasets, as demonstrated by Adobe's results.
2
Integrating AI model training within the same Spark cluster can streamline workflows.
This integration reduces the complexity of managing separate infrastructures, allowing for more efficient data processing and model training.
3
Leverage the RAPIDS software suite for optimized performance in data science tasks.
Using RAPIDS can significantly enhance the performance of Spark applications, making it a valuable tool for data scientists across various industries.