  In this paper, we introduce neural texture learning for 6D object pose
estimation from synthetic data and a few unlabelled real images. Our major
contribution is a novel learning scheme which removes the drawbacks of previous
works, namely the strong dependency on co-modalities or additional refinement.
These have been previously necessary to provide training signals for
convergence. We formulate such a scheme as two sub-optimisation problems on
texture learning and pose learning. We separately learn to predict realistic
texture of objects from real image collections and learn pose estimation from
pixel-perfect synthetic data. Combining these two capabilities allows then to
synthesise photorealistic novel views to supervise the pose estimator with
accurate geometry. To alleviate pose noise and segmentation imperfection
present during the texture learning phase, we propose a surfel-based
adversarial training loss together with texture regularisation from synthetic
data. We demonstrate that the proposed approach significantly outperforms the
recent state-of-the-art methods without ground-truth pose annotations and
demonstrates substantial generalisation improvements towards unseen scenes.
Remarkably, our scheme improves the adopted pose estimators substantially even
when initialised with much inferior performance.
