SERL: A Software Suite for Sample-Efficient Robotic Reinforcement
  Learning
In recent years, significant progress has been made in the field of robotic
reinforcement learning (RL), enabling methods that handle complex image
observations, train in the real world, and incorporate auxiliary data, such as
demonstrations and prior experience. However, despite these advances, robotic
RL remains hard to use. It is acknowledged among practitioners that the
particular implementation details of these algorithms are often just as
important (if not more so) for performance as the choice of algorithm. We posit
that a significant challenge to widespread adoption of robotic RL, as well as
further development of robotic RL methods, is the comparative inaccessibility
of such methods. To address this challenge, we developed a carefully
implemented library containing a sample efficient off-policy deep RL method,
together with methods for computing rewards and resetting the environment, a
high-quality controller for a widely-adopted robot, and a number of challenging
example tasks. We provide this library as a resource for the community,
describe its design choices, and present experimental results. Perhaps
surprisingly, we find that our implementation can achieve very efficient
learning, acquiring policies for PCB board assembly, cable routing, and object
relocation between 25 to 50 minutes of training per policy on average,
improving over state-of-the-art results reported for similar tasks in the
literature. These policies achieve perfect or near-perfect success rates,
extreme robustness even under perturbations, and exhibit emergent recovery and
correction behaviors. We hope that these promising results and our high-quality
open-source implementation will provide a tool for the robotics community to
facilitate further developments in robotic RL. Our code, documentation, and
videos can be found at https://serl-robot.github.io/
65b88b1954cfdff8f8814022
Jianlan Luo
claimed_verified
2024-01-30T08:06:13.647Z
64f8cb8ed04a890f5380d9a4
/avatars/d6fdfdbb0c10141aa3b4c832d928121b.svg
Jianlan Luo
user
jianlanluo
65b88b1954cfdff8f8814023
Zheyuan Hu
65b88b1954cfdff8f8814024
Charles Xu
admin_assigned
2024-01-30T09:46:26.033Z
658fdaacac02633c0d78e3e8
/avatars/c63481494f86368185a9e33587de77d3.svg
Charles Xu
user
charlesxu0124
65b88b1954cfdff8f8814025
You Liang Tan
admin_assigned
2024-01-30T09:46:32.072Z
635dcbee79599a4f7db46db3
https://cdn-avatars.huggingface.co/v1/production/uploads/635dcbee79599a4f7db46db3/eIi5cPQelHi0TkNNe1_XT.jpeg
You Liang Tan
user
youliangtan
65b88b1954cfdff8f8814026
Jacob Berg
65b88b1954cfdff8f8814027
Archit Sharma
65b88b1954cfdff8f8814028
Stefan Schaal
65b88b1954cfdff8f8814029
Chelsea Finn
admin_assigned
2024-01-30T09:47:29.290Z
64ac22a9193f0a807deb673d
/avatars/fcac4912678ad3cb6e817d40bdee9aea.svg
Chelsea Finn
user
cbfinn
65b88b1954cfdff8f881402a
Abhishek Gupta
65b88b1954cfdff8f881402b
Sergey Levine
SERL: A Software Suite for Sample-Efficient Robotic Reinforcement
  Learning
In recent years, significant progress has been made in the field of robotic
reinforcement learning (RL), enabling methods that handle complex image
observations, train in the real world, and incorporate auxiliary data, such as
demonstrations and prior experience. However, despite these advances, robotic
RL remains hard to use. It is acknowledged among practitioners that the
particular implementation details of these algorithms are often just as
important (if not more so) for performance as the choice of algorithm. We posit
that a significant challenge to widespread adoption of robotic RL, as well as
further development of robotic RL methods, is the comparative inaccessibility
of such methods. To address this challenge, we developed a carefully
implemented library containing a sample efficient off-policy deep RL method,
together with methods for computing rewards and resetting the environment, a
high-quality controller for a widely-adopted robot, and a number of challenging
example tasks. We provide this library as a resource for the community,
describe its design choices, and present experimental results. Perhaps
surprisingly, we find that our implementation can achieve very efficient
learning, acquiring policies for PCB board assembly, cable routing, and object
relocation between 25 to 50 minutes of training per policy on average,
improving over state-of-the-art results reported for similar tasks in the
literature. These policies achieve perfect or near-perfect success rates,
extreme robustness even under perturbations, and exhibit emergent recovery and
correction behaviors. We hope that these promising results and our high-quality
open-source implementation will provide a tool for the robotics community to
facilitate further developments in robotic RL. Our code, documentation, and
videos can be found at https://serl-robot.github.io/
A library for robotic reinforcement learning provides efficient off-policy deep RL methods, high-quality controllers, and example tasks, achieving state-of-the-art results in tasks like PCB board assembly and cable routing.
off-policy deep RL rewards environment reset high-quality controller emergent recovery