  3D surface reconstruction from images is essential for numerous applications.
Recently, Neural Radiance Fields (NeRFs) have emerged as a promising framework
for 3D modeling. However, NeRFs require accurate camera poses as input, and
existing methods struggle to handle significantly noisy pose estimates (i.e.,
outliers), which are commonly encountered in real-world scenarios. To tackle
this challenge, we present a novel approach that optimizes radiance fields with
scene graphs to mitigate the influence of outlier poses. Our method
incorporates an adaptive inlier-outlier confidence estimation scheme based on
scene graphs, emphasizing images of high compatibility with the neighborhood
and consistency in the rendering quality. We also introduce an effective
intersection-over-union (IoU) loss to optimize the camera pose and surface
geometry, together with a coarse-to-fine strategy to facilitate the training.
Furthermore, we propose a new dataset containing typical outlier poses for a
detailed evaluation. Experimental results on various datasets consistently
demonstrate the effectiveness and superiority of our method over existing
approaches, showcasing its robustness in handling outliers and producing
high-quality 3D reconstructions. Our code and data are available at:
\url{https://github.com/Iris-cyy/SG-NeRF}.
