  This paper presents a framework that leverages both control theory and
machine learning to obtain stable and robust bipedal locomotion without the
need for manual parameter tuning. Traditionally, gaits are generated through
trajectory optimization methods and then realized experimentally -- a process
that often requires extensive tuning due to differences between the models and
hardware. In this work, the process of gait realization via hybrid zero
dynamics (HZD) based optimization is formally combined with preference-based
learning to systematically realize dynamically stable walking. Importantly,
this learning approach does not require a carefully constructed reward
function, but instead utilizes human pairwise preferences. The power of the
proposed approach is demonstrated through two experiments on a planar biped
AMBER-3M: the first with rigid point-feet, and the second with induced model
uncertainty through the addition of springs where the added compliance was not
accounted for in the gait generation or in the controller. In both experiments,
the framework achieves stable, robust, efficient, and natural walking in fewer
than 50 iterations with no reliance on a simulation environment. These results
demonstrate a promising step in the unification of control theory and learning.
