Sparse Forests with FIL

Andy Adinets

The RAPIDS Forest Inference Library, affectionately known as FIL, dramatically accelerates inference (prediction) for tree-based models…

NVIDIA

•

Andy Adinets

•5 min read•intermediate•

--

•View Original

LightGBMPythonscikit-learnXGBoost

Overview

The article discusses the RAPIDS Forest Inference Library (FIL) and its support for sparse tree storage, which significantly reduces memory usage for deep tree-based models. It provides guidance on using sparse forests with FIL, implementation details, benchmarking results, and concludes with the library's advantages for various machine learning tasks.

What You'll Learn

1

How to use the storage_type parameter to create sparse forests in FIL

2

Why sparse tree storage is beneficial for deep tree models

3

How to benchmark inference performance using FIL with sparse forests

Prerequisites & Requirements

Understanding of tree-based models like XGBoost and LightGBM
Familiarity with RAPIDS AI and its libraries(optional)

Key Questions Answered

How does sparse tree storage in FIL save GPU memory?

Sparse tree storage in FIL allocates memory only for existing nodes, meaning if a branch ends before the maximum depth, no memory is wasted on potential children. For example, a dense tree of depth 30 requires over 2 billion nodes, while a sparse tree can require as few as 61 nodes, leading to significant memory savings.

What are the performance differences between dense and sparse FIL predictors?

Dense FIL predictors are faster for shallow forests but can run out of memory for deep trees, while sparse predictors handle deep trees efficiently. Benchmarks show that FIL predictors are about 34–60x faster than SKLearn CPU predictors, making them a better choice for deep models.

What parameters control the type of forest created in FIL?

The type of forest in FIL is controlled by the storage_type parameter in ForestInference.load(). It can be set to DENSE, SPARSE, or AUTO, with AUTO currently defaulting to DENSE. This flexibility allows users to optimize memory usage based on their model's depth.

Key Statistics & Figures

Inference speed improvement

34–60x faster

FIL predictors compared to SKLearn CPU predictors

Maximum depth of dense trees before memory issues

30-50

Dense FIL forests run out of memory beyond this depth

Technologies & Tools

Some links below are affiliate links. We may earn a commission if you make a purchase.

Library

Rapids AI

Used for accelerating inference for tree-based models

Library

Xgboost

Model training for tree-based models

Library

Lightgbm

Model training for tree-based models

Library

Scikit-learn

Model training and data generation for benchmarking

Key Actionable Insights

1
Utilize the sparse forest option in FIL for deep tree models to save GPU memory and improve inference speed.
When working with deep tree-based models, switching to sparse forests can prevent memory overflow issues and enhance performance, especially when dealing with large datasets.

2
Benchmark your models using FIL to compare inference times against traditional CPU methods.
By measuring inference performance on both GPU and CPU, you can make informed decisions about which model and storage type to use for optimal efficiency in production environments.

Common Pitfalls

1

Assuming dense forests are always the best choice for all tree depths.

Dense forests can lead to memory issues with deep trees, while sparse forests are designed to handle such cases efficiently. It's crucial to evaluate the depth of your trees and choose the appropriate storage type.

Related Concepts

Tree-based Models

Memory Optimization Techniques

Inference Performance Benchmarking