  The use of multiple camera technologies in a combined multimodal monitoring
system for plant phenotyping offers promising benefits. Compared to
configurations that only utilize a single camera technology, cross-modal
patterns can be recorded that allow a more comprehensive assessment of plant
phenotypes. However, the effective utilization of cross-modal patterns is
dependent on precise image registration to achieve pixel-accurate alignment, a
challenge often complicated by parallax and occlusion effects inherent in plant
canopy imaging.
  In this study, we propose a novel multimodal 3D image registration method
that addresses these challenges by integrating depth information from a
time-of-flight camera into the registration process. By leveraging depth data,
our method mitigates parallax effects and thus facilitates more accurate pixel
alignment across camera modalities. Additionally, we introduce an automated
mechanism to identify and differentiate different types of occlusions, thereby
minimizing the introduction of registration errors.
  To evaluate the efficacy of our approach, we conduct experiments on a diverse
image dataset comprising six distinct plant species with varying leaf
geometries. Our results demonstrate the robustness of the proposed registration
algorithm, showcasing its ability to achieve accurate alignment across
different plant types and camera compositions. Compared to previous methods it
is not reliant on detecting plant specific image features and can thereby be
utilized for a wide variety of applications in plant sciences. The registration
approach principally scales to arbitrary numbers of cameras with different
resolutions and wavelengths. Overall, our study contributes to advancing the
field of plant phenotyping by offering a robust and reliable solution for
multimodal image registration.
