  The motivation of our work is to present a new visualization-guided computing
paradigm to combine direct 3D volume processing and volume rendered clues for
effective 3D exploration such as extracting and visualizing microstructures
in-vivo. However, it is still challenging to extract and visualize high
fidelity 3D vessel structure due to its high sparseness, noisiness, and complex
topology variations. In this paper, we present an end-to-end deep learning
method, VC-Net, for robust extraction of 3D microvasculature through embedding
the image composition, generated by maximum intensity projection (MIP), into 3D
volume image learning to enhance the performance. The core novelty is to
automatically leverage the volume visualization technique (MIP) to enhance the
3D data exploration at deep learning level. The MIP embedding features can
enhance the local vessel signal and are adaptive to the geometric variability
and scalability of vessels, which is crucial in microvascular tracking. A
multi-stream convolutional neural network is proposed to learn the 3D volume
and 2D MIP features respectively and then explore their inter-dependencies in a
joint volume-composition embedding space by unprojecting the MIP features into
3D volume embedding space. The proposed framework can better capture small /
micro vessels and improve vessel connectivity. To our knowledge, this is the
first deep learning framework to construct a joint convolutional embedding
space, where the computed vessel probabilities from volume rendering based 2D
projection and 3D volume can be explored and integrated synergistically.
Experimental results are compared with the traditional 3D vessel segmentation
methods and the deep learning state-of-the-art on public and real patient
(micro-)cerebrovascular image datasets. Our method demonstrates the potential
in a powerful MR arteriogram and venogram diagnosis of vascular diseases.
