  In this work, we introduce a control framework that combines model-based
footstep planning with Reinforcement Learning (RL), leveraging desired footstep
patterns derived from the Linear Inverted Pendulum (LIP) dynamics. Utilizing
the LIP model, our method forward predicts robot states and determines the
desired foot placement given the velocity commands. We then train an RL policy
to track the foot placements without following the full reference motions
derived from the LIP model. This partial guidance from the physics model allows
the RL policy to integrate the predictive capabilities of the physics-informed
dynamics and the adaptability characteristics of the RL controller without
overfitting the policy to the template model. Our approach is validated on the
MIT Humanoid, demonstrating that our policy can achieve stable yet dynamic
locomotion for walking and turning. We further validate the adaptability and
generalizability of our policy by extending the locomotion task to unseen,
uneven terrain. During the hardware deployment, we have achieved forward
walking speeds of up to 1.5 m/s on a treadmill and have successfully performed
dynamic locomotion maneuvers such as 90-degree and 180-degree turns.
