  Similar to their counterparts in nature, the flexible bodies of snake-like
robots enhance their movement capability and adaptability in diverse
environments. However, this flexibility corresponds to a complex control task
involving highly redundant degrees of freedom, where traditional model-based
methods usually fail to propel the robots energy-efficiently. In this work, we
present a novel approach for designing an energy-efficient slithering gait for
a snake-like robot using a model-free reinforcement learning (RL) algorithm.
Specifically, we present an RL-based controller for generating locomotion gaits
at a wide range of velocities, which is trained using the proximal policy
optimization (PPO) algorithm. Meanwhile, a traditional parameterized gait
controller is presented and the parameter sets are optimized using the grid
search and Bayesian optimization algorithms for the purposes of reasonable
comparisons. Based on the analysis of the simulation results, we demonstrate
that this RL-based controller exhibits very natural and adaptive movements,
which are also substantially more energy-efficient than the gaits generated by
the parameterized controller. Videos are shown at
\textcolor{blue}{\href{https://videoviewsite.wixsite.com/rlsnake}{https://videoviewsite.wixsite.com/rlsnake}}.
