  Mutual calibration between color and depth cameras is a challenging topic in
multi-modal data registration. In this paper, we are confronted with a "Bimodal
Stereo" problem, which aims to solve camera pose from a pair of an uncalibrated
color image and a depth map from different views automatically. To address this
problem, an iterative Shape-from-Shading (SfS) based framework is proposed to
estimate shape and pose simultaneously. In the pipeline, the estimated shape is
refined by the shape prior from the given depth map under the estimated pose.
Meanwhile, the estimated pose is improved by the registration of estimated
shape and shape from given depth map. We also introduce a shading based
refinement in the pipeline to address noisy depth map with holes. Extensive
experiments showed that through our method, both the depth map, the recovered
shape as well as its pose can be desirably refined and recovered.
