  Automatic and accurate detection of anatomical landmarks is an essential
operation in medical image analysis with a multitude of applications. Recent
deep learning methods have improved results by directly encoding the appearance
of the captured anatomy with the likelihood maps (i.e., heatmaps). However,
most current solutions overlook another essence of heatmap regression, the
objective metric for regressing target heatmaps and rely on hand-crafted
heuristics to set the target precision, thus being usually cumbersome and
task-specific. In this paper, we propose a novel learning-to-learn framework
for landmark detection to optimize the neural network and the target precision
simultaneously. The pivot of this work is to leverage the reinforcement
learning (RL) framework to search objective metrics for regressing multiple
heatmaps dynamically during the training process, thus avoiding setting
problem-specific target precision. We also introduce an early-stop strategy for
active termination of the RL agent's interaction that adapts the optimal
precision for separate targets considering exploration-exploitation tradeoffs.
This approach shows better stability in training and improved localization
accuracy in inference. Extensive experimental results on two different
applications of landmark localization: 1) our in-house prenatal ultrasound (US)
dataset and 2) the publicly available dataset of cephalometric X-Ray landmark
detection, demonstrate the effectiveness of our proposed method. Our proposed
framework is general and shows the potential to improve the efficiency of
anatomical landmark detection.
