  Humans excel at robust bipedal walking in complex natural environments. In
each step, they adequately tune the interaction of biomechanical muscle
dynamics and neuronal signals to be robust against uncertainties in ground
conditions. However, it is still not fully understood how the nervous system
resolves the musculoskeletal redundancy to solve the multi-objective control
problem considering stability, robustness, and energy efficiency. In computer
simulations, energy minimization has been shown to be a successful optimization
target, reproducing natural walking with trajectory optimization or
reflex-based control methods. However, these methods focus on particular
motions at a time and the resulting controllers are limited when compensating
for perturbations. In robotics, reinforcement learning~(RL) methods recently
achieved highly stable (and efficient) locomotion on quadruped systems, but the
generation of human-like walking with bipedal biomechanical models has required
extensive use of expert data sets. This strong reliance on demonstrations often
results in brittle policies and limits the application to new behaviors,
especially considering the potential variety of movements for high-dimensional
musculoskeletal models in 3D. Achieving natural locomotion with RL without
sacrificing its incredible robustness might pave the way for a novel approach
to studying human walking in complex natural environments. Videos:
https://sites.google.com/view/naturalwalkingrl
