  In this work, we introduce the Geometry-Aware Large Reconstruction Model
(GeoLRM), an approach which can predict high-quality assets with 512k Gaussians
and 21 input images in only 11 GB GPU memory. Previous works neglect the
inherent sparsity of 3D structure and do not utilize explicit geometric
relationships between 3D and 2D images. This limits these methods to a
low-resolution representation and makes it difficult to scale up to the dense
views for better quality. GeoLRM tackles these issues by incorporating a novel
3D-aware transformer structure that directly processes 3D points and uses
deformable cross-attention mechanisms to effectively integrate image features
into 3D representations. We implement this solution through a two-stage
pipeline: initially, a lightweight proposal network generates a sparse set of
3D anchor points from the posed image inputs; subsequently, a specialized
reconstruction transformer refines the geometry and retrieves textural details.
Extensive experimental results demonstrate that GeoLRM significantly
outperforms existing models, especially for dense view inputs. We also
demonstrate the practical applicability of our model with 3D generation tasks,
showcasing its versatility and potential for broader adoption in real-world
applications.
