  Heatmap regression with a deep network has become one of the mainstream
approaches to localize facial landmarks. However, the loss function for heatmap
regression is rarely studied. In this paper, we analyze the ideal loss function
properties for heatmap regression in face alignment problems. Then we propose a
novel loss function, named Adaptive Wing loss, that is able to adapt its shape
to different types of ground truth heatmap pixels. This adaptability penalizes
loss more on foreground pixels while less on background pixels. To address the
imbalance between foreground and background pixels, we also propose Weighted
Loss Map, which assigns high weights on foreground and difficult background
pixels to help training process focus more on pixels that are crucial to
landmark localization. To further improve face alignment accuracy, we introduce
boundary prediction and CoordConv with boundary coordinates. Extensive
experiments on different benchmarks, including COFW, 300W and WFLW, show our
approach outperforms the state-of-the-art by a significant margin on various
evaluation metrics. Besides, the Adaptive Wing loss also helps other heatmap
regression tasks. Code will be made publicly available at
https://github.com/protossw512/AdaptiveWingLoss.
