  Neural Radiance Fields (NeRF) use multi-view images for 3D scene
representation, demonstrating remarkable performance. As one of the primary
sources of multi-view images, multi-camera systems encounter challenges such as
varying intrinsic parameters and frequent pose changes. Most previous
NeRF-based methods assume a unique camera and rarely consider multi-camera
scenarios. Besides, some NeRF methods that can optimize intrinsic and extrinsic
parameters still remain susceptible to suboptimal solutions when these
parameters are poor initialized. In this paper, we propose MC-NeRF, a method
that enables joint optimization of both intrinsic and extrinsic parameters
alongside NeRF. The method also supports each image corresponding to
independent camera parameters. First, we tackle coupling issue and the
degenerate case that arise from the joint optimization between intrinsic and
extrinsic parameters. Second, based on the proposed solutions, we introduce an
efficient calibration image acquisition scheme for multi-camera systems,
including the design of calibration object. Finally, we present an end-to-end
network with training sequence that enables the estimation of intrinsic and
extrinsic parameters, along with the rendering network. Furthermore,
recognizing that most existing datasets are designed for a unique camera, we
construct a real multi-camera image acquisition system and create a
corresponding new dataset, which includes both simulated data and real-world
captured images. Experiments confirm the effectiveness of our method when each
image corresponds to different camera parameters. Specifically, we use
multi-cameras, each with different intrinsic and extrinsic parameters in
real-world system, to achieve 3D scene representation without providing initial
poses.
