  In this work, we propose a camera self-calibration algorithm for generic
cameras with arbitrary non-linear distortions. We jointly learn the geometry of
the scene and the accurate camera parameters without any calibration objects.
Our camera model consists of a pinhole model, a fourth order radial distortion,
and a generic noise model that can learn arbitrary non-linear camera
distortions. While traditional self-calibration algorithms mostly rely on
geometric constraints, we additionally incorporate photometric consistency.
This requires learning the geometry of the scene, and we use Neural Radiance
Fields (NeRF). We also propose a new geometric loss function, viz., projected
ray distance loss, to incorporate geometric consistency for complex non-linear
camera models. We validate our approach on standard real image datasets and
demonstrate that our model can learn the camera intrinsics and extrinsics
(pose) from scratch without COLMAP initialization. Also, we show that learning
accurate camera models in a differentiable manner allows us to improve PSNR
over baselines. Our module is an easy-to-use plugin that can be applied to NeRF
variants to improve performance. The code and data are currently available at
https://github.com/POSTECH-CVLab/SCNeRF.
