  Estimating the 3DoF rotation from a single RGB image is an important yet
challenging problem. Recent works achieve good performance relying on a large
amount of expensive-to-obtain labeled data. To reduce the amount of
supervision, we for the first time propose a general framework, FisherMatch,
for semi-supervised rotation regression, without assuming any domain-specific
knowledge or paired data. Inspired by the popular semi-supervised approach,
FixMatch, we propose to leverage pseudo label filtering to facilitate the
information flow from labeled data to unlabeled data in a teacher-student
mutual learning framework. However, incorporating the pseudo label filtering
mechanism into semi-supervised rotation regression is highly non-trivial,
mainly due to the lack of a reliable confidence measure for rotation
prediction. In this work, we propose to leverage matrix Fisher distribution to
build a probabilistic model of rotation and devise a matrix Fisher-based
regressor for jointly predicting rotation along with its prediction
uncertainty. We then propose to use the entropy of the predicted distribution
as a confidence measure, which enables us to perform pseudo label filtering for
rotation regression. For supervising such distribution-like pseudo labels, we
further investigate the problem of how to enforce loss between two matrix
Fisher distributions. Our extensive experiments show that our method can work
well even under very low labeled data ratios on different benchmarks, achieving
significant and consistent performance improvement over supervised learning and
other semi-supervised learning baselines. Our project page is at
https://yd-yin.github.io/FisherMatch.
