  This paper addresses the problem of viewpoint estimation of an object in a
given image. It presents five key insights that should be taken into
consideration when designing a CNN that solves the problem. Based on these
insights, the paper proposes a network in which (i) The architecture jointly
solves detection, classification, and viewpoint estimation. (ii) New types of
data are added and trained on. (iii) A novel loss function, which takes into
account both the geometry of the problem and the new types of data, is propose.
Our network improves the state-of-the-art results for this problem by 9.8%.
