  Text-to-image diffusion models generate impressive and realistic images, but
do they learn to represent the 3D world from only 2D supervision? We
demonstrate that yes, certain 3D scene representations are encoded in the text
embedding space of models like Stable Diffusion. Our approach, Viewpoint Neural
Textual Inversion (ViewNeTI), is to discover 3D view tokens; these tokens
control the 3D viewpoint - the rendering pose in a scene - of generated images.
Specifically, we train a small neural mapper to take continuous camera
viewpoint parameters and predict a view token (a word embedding). This token
conditions diffusion generation via cross-attention to produce images with the
desired camera viewpoint. Using ViewNeTI as an evaluation tool, we report two
findings: first, the text latent space has a continuous view-control manifold
for particular 3D scenes; second, we find evidence for a generalized
view-control manifold for all scenes. We conclude that since the view token
controls the 3D `rendering' viewpoint, there is likely a scene representation
embedded in frozen 2D diffusion models. Finally, we exploit the 3D scene
representations for 3D vision tasks, namely, view-controlled text-to-image
generation, and novel view synthesis from a single image, where our approach
sets state-of-the-art for LPIPS. Code available at
https://github.com/jmhb0/view_neti
