  This paper presents a normalization mechanism called Instance-Level Meta
Normalization (ILM~Norm) to address a learning-to-normalize problem. ILM~Norm
learns to predict the normalization parameters via both the feature
feed-forward and the gradient back-propagation paths. ILM~Norm provides a meta
normalization mechanism and has several good properties. It can be easily
plugged into existing instance-level normalization schemes such as Instance
Normalization, Layer Normalization, or Group Normalization. ILM~Norm normalizes
each instance individually and therefore maintains high performance even when
small mini-batch is used. The experimental results show that ILM~Norm well
adapts to different network architectures and tasks, and it consistently
improves the performance of the original models. The code is available at
url{https://github.com/Gasoonjia/ILM-Norm.
