  We present in this paper ByteCover, which is a new feature learning method
for cover song identification (CSI). ByteCover is built based on the classical
ResNet model, and two major improvements are designed to further enhance the
capability of the model for CSI. In the first improvement, we introduce the
integration of instance normalization (IN) and batch normalization (BN) to
build IBN blocks, which are major components of our ResNet-IBN model. With the
help of the IBN blocks, our CSI model can learn features that are invariant to
the changes of musical attributes such as key, tempo, timbre and genre, while
preserving the version information. In the second improvement, we employ the
BNNeck method to allow a multi-loss training and encourage our method to
jointly optimize a classification loss and a triplet loss, and by this means,
the inter-class discrimination and intra-class compactness of cover songs, can
be ensured at the same time. A set of experiments demonstrated the
effectiveness and efficiency of ByteCover on multiple datasets, and in the
Da-TACOS dataset, ByteCover outperformed the best competitive system by 20.9\%.
