Gradient Boosting Reinforcement Learning
Neural networks (NN) achieve remarkable results in various tasks, but lack
key characteristics: interpretability, support for categorical features, and
lightweight implementations suitable for edge devices. While ongoing efforts
aim to address these challenges, Gradient Boosting Trees (GBT) inherently meet
these requirements. As a result, GBTs have become the go-to method for
supervised learning tasks in many real-world applications and competitions.
However, their application in online learning scenarios, notably in
reinforcement learning (RL), has been limited. In this work, we bridge this gap
by introducing Gradient-Boosting RL (GBRL), a framework that extends the
advantages of GBT to the RL domain. Using the GBRL framework, we implement
various actor-critic algorithms and compare their performance with their NN
counterparts. Inspired by shared backbones in NN we introduce a tree-sharing
approach for policy and value functions with distinct learning rates, enhancing
learning efficiency over millions of interactions. GBRL achieves competitive
performance across a diverse array of tasks, excelling in domains with
structured or categorical features. Additionally, we present a
high-performance, GPU-accelerated implementation that integrates seamlessly
with widely-used RL libraries (available at https://github.com/NVlabs/gbrl).
GBRL expands the toolkit for RL practitioners, demonstrating the viability and
promise of GBT within the RL paradigm, particularly in domains characterized by
structured or categorical features.
66908cf75cb4c07caa1f91c2
Benjamin Fuhrer
extracted_pending
2024-07-12T01:55:08.622Z
64a58d68f95d48e7c521008d
/avatars/64fdf06190064623e16fadbc3965a847.svg
benjamin fuhrer
user
bfuhrer
66908cf75cb4c07caa1f91c3
Chen Tessler
extracted_confirmed
2025-03-18T21:31:51.065Z
67d9db8a8d942645bf063484
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/7JeFDO4Ky7XjTCRL4ROBl.png
Chen Tessler
user
ctessler
66908cf75cb4c07caa1f91c4
Gal Dalal
claimed_verified
2024-07-15T13:34:43.925Z
66952220948a73be7f51d4ff
/avatars/1f6cfe7502b2cb697816caa3c5334c7b.svg
Gal Dalal
user
GalDalal
Gradient Boosting Reinforcement Learning
Neural networks (NN) achieve remarkable results in various tasks, but lack
key characteristics: interpretability, support for categorical features, and
lightweight implementations suitable for edge devices. While ongoing efforts
aim to address these challenges, Gradient Boosting Trees (GBT) inherently meet
these requirements. As a result, GBTs have become the go-to method for
supervised learning tasks in many real-world applications and competitions.
However, their application in online learning scenarios, notably in
reinforcement learning (RL), has been limited. In this work, we bridge this gap
by introducing Gradient-Boosting RL (GBRL), a framework that extends the
advantages of GBT to the RL domain. Using the GBRL framework, we implement
various actor-critic algorithms and compare their performance with their NN
counterparts. Inspired by shared backbones in NN we introduce a tree-sharing
approach for policy and value functions with distinct learning rates, enhancing
learning efficiency over millions of interactions. GBRL achieves competitive
performance across a diverse array of tasks, excelling in domains with
structured or categorical features. Additionally, we present a
high-performance, GPU-accelerated implementation that integrates seamlessly
with widely-used RL libraries (available at https://github.com/NVlabs/gbrl).
GBRL expands the toolkit for RL practitioners, demonstrating the viability and
promise of GBT within the RL paradigm, particularly in domains characterized by
structured or categorical features.
Gradient-Boosting RL (GBRL) enhances reinforcement learning with the strengths of gradient boosting trees, providing efficient and competitive performance in tasks with structured or categorical features.
Gradient-Boosting RL GBRL actor-critic algorithms tree-sharing learning efficiency GPU-accelerated implementation