  The cross-entropy softmax loss is the primary loss function used to train
deep neural networks. On the other hand, the focal loss function has been
demonstrated to provide improved performance when there is an imbalance in the
number of training samples in each class, such as in long-tailed datasets. In
this paper, we introduce a novel cyclical focal loss and demonstrate that it is
a more universal loss function than cross-entropy softmax loss or focal loss.
We describe the intuition behind the cyclical focal loss and our experiments
provide evidence that cyclical focal loss provides superior performance for
balanced, imbalanced, or long-tailed datasets. We provide numerous experimental
results for CIFAR-10/CIFAR-100, ImageNet, balanced and imbalanced 4,000
training sample versions of CIFAR-10/CIFAR-100, and ImageNet-LT and Places-LT
from the Open Long-Tailed Recognition (OLTR) challenge. Implementing the
cyclical focal loss function requires only a few lines of code and does not
increase training time. In the spirit of reproducibility, our code is available
at \url{https://github.com/lnsmith54/CFL}.
