  Pre-training on large-scale databases consisting of natural images and then
fine-tuning them to fit the application at hand, or transfer-learning, is a
popular strategy in computer vision. However, Kataoka et al., 2020 introduced a
technique to eliminate the need for natural images in supervised deep learning
by proposing a novel synthetic, formula-based method to generate 2D fractals as
training corpus. Using one synthetically generated fractal for each class, they
achieved transfer learning results comparable to models pre-trained on natural
images. In this project, we take inspiration from their work and build on this
idea -- using 3D procedural object renders. Since the image formation process
in the natural world is based on its 3D structure, we expect pre-training with
3D mesh renders to provide an implicit bias leading to better generalization
capabilities in a transfer learning setting and that invariances to 3D rotation
and illumination are easier to be learned based on 3D data. Similar to the
previous work, our training corpus will be fully synthetic and derived from
simple procedural strategies; we will go beyond classic data augmentation and
also vary illumination and pose which are controllable in our setting and study
their effect on transfer learning capabilities in context to prior work. In
addition, we will compare the 2D fractal and 3D procedural object networks to
human and non-human primate brain data to learn more about the 2D vs. 3D nature
of biological vision.
