  Humanoid robots are made to resemble humans but their locomotion abilities
are far from ours in terms of agility and versatility. When humans walk on
complex terrains, or face external disturbances, they combine a set of
strategies, unconsciously and efficiently, to regain stability. This paper
tackles the problem of developing a robust omnidirectional walking framework,
which is able to generate versatile and agile locomotion on complex terrains.
The Linear Inverted Pendulum Model and Central Pattern Generator concepts are
used to develop a closed-loop walk engine, which is then combined with a
reinforcement learning module. This module learns to regulate the walk engine
parameters adaptively, and generates residuals to adjust the robot's target
joint positions (residual physics). Additionally, we propose a proximal
symmetry loss function to increase the sample efficiency of the Proximal Policy
Optimization algorithm, by leveraging model symmetries and the trust region
concept. The effectiveness of the proposed framework was demonstrated and
evaluated across a set of challenging simulation scenarios. The robot was able
to generalize what it learned in unforeseen circumstances, displaying
human-like locomotion skills, even in the presence of noise and external
pushes.
