  Safety has been a critical issue for the deployment of learning-based
approaches in real-world applications. To address this issue, control barrier
function (CBF) and its variants have attracted extensive attention for
safety-critical control. However, due to the myopic one-step nature of CBF and
the lack of principled methods to design the class-$\mathcal{K}$ functions,
there are still fundamental limitations of current CBFs: optimality, stability,
and feasibility. In this paper, we proposed a novel and unified approach to
address these limitations with Adaptive Multi-step Control Barrier Function
(AM-CBF), where we parameterize the class-$\mathcal{K}$ function by a neural
network and train it together with the reinforcement learning policy. Moreover,
to mitigate the myopic nature, we propose a novel \textit{multi-step training
and single-step execution} paradigm to make CBF farsighted while the execution
remains solving a single-step convex quadratic program. Our method is evaluated
on the first and second-order systems in various scenarios, where our approach
outperforms the conventional CBF both qualitatively and quantitatively.
