  Colorization involves the synthesis of colors on a target image while
preserving structural content as well as the semantics of the target image.
This is a well-explored problem in 2D with many state-of-the-art solutions. We
propose a novel deep learning-based approach for the colorization of 3D medical
volumes. Our system is capable of directly mapping the colors of a 2D
photograph to a 3D MRI volume in real-time, producing a high-fidelity color
volume suitable for photo-realistic visualization. Since this work is first of
its kind, we discuss the full pipeline in detail and the challenges that it
brings for 3D medical data. The colorization of medical MRI volume also entails
modality conversion that highlights the robustness of our approach in handling
multi-modal data.
