  Object pose increases intraclass object variance which makes object
recognition from 2D images harder. To render a classifier robust to pose
variations, most deep neural networks try to eliminate the influence of pose by
using large datasets with many poses for each class. Here, we propose a
different approach: a class-agnostic object pose transformation network
(OPT-Net) can transform an image along 3D yaw and pitch axes to synthesize
additional poses continuously. Synthesized images lead to better training of an
object classifier. We design a novel eliminate-add structure to explicitly
disentangle pose from object identity: first eliminate pose information of the
input image and then add target pose information (regularized as continuous
variables) to synthesize any target pose. We trained OPT-Net on images of toy
vehicles shot on a turntable from the iLab-20M dataset. After training on
unbalanced discrete poses (5 classes with 6 poses per object instance, plus 5
classes with only 2 poses), we show that OPT-Net can synthesize balanced
continuous new poses along yaw and pitch axes with high quality. Training a
ResNet-18 classifier with original plus synthesized poses improves mAP accuracy
by 9% overtraining on original poses only. Further, the pre-trained OPT-Net can
generalize to new object classes, which we demonstrate on both iLab-20M and
RGB-D. We also show that the learned features can generalize to ImageNet.
