  Reinforcement learning is applied to solve actual complex tasks from
high-dimensional, sensory inputs. The last decade has developed a long list of
reinforcement learning algorithms. Recent progress benefits from deep learning
for raw sensory signal representation. One question naturally arises: how well
do they perform concerning different robotic manipulation tasks? Benchmarks use
objective performance metrics to offer a scientific way to compare algorithms.
In this paper, we present RMBench, the first benchmark for robotic
manipulations, which have high-dimensional continuous action and state spaces.
We implement and evaluate reinforcement learning algorithms that directly use
observed pixels as inputs. We report their average performance and learning
curves to show their performance and stability of training. Our study concludes
that none of the studied algorithms can handle all tasks well, soft
Actor-Critic outperforms most algorithms in average reward and stability, and
an algorithm combined with data augmentation may facilitate learning policies.
Our code is publicly available at
https://github.com/xiangyanfei212/RMBench-2022, including all benchmark tasks
and studied algorithms.
