  Deformable shapes provide important and complex geometric features of objects
presented in images. However, such information is oftentimes missing or
underutilized as implicit knowledge in many image analysis tasks. This paper
presents Geo-SIC, the first deep learning model to learn deformable shapes in a
deformation space for an improved performance of image classification. We
introduce a newly designed framework that (i) simultaneously derives features
from both image and latent shape spaces with large intra-class variations; and
(ii) gains increased model interpretability by allowing direct access to the
underlying geometric features of image data. In particular, we develop a
boosted classification network, equipped with an unsupervised learning of
geometric shape representations characterized by diffeomorphic transformations
within each class. In contrast to previous approaches using pre-extracted
shapes, our model provides a more fundamental approach by naturally learning
the most relevant shape features jointly with an image classifier. We
demonstrate the effectiveness of our method on both simulated 2D images and
real 3D brain magnetic resonance (MR) images. Experimental results show that
our model substantially improves the image classification accuracy with an
additional benefit of increased model interpretability. Our code is publicly
available at https://github.com/jw4hv/Geo-SIC
