  Accurate volume segmentation from the Computed Tomography (CT) scan is a
common prerequisite for pre-operative planning, intra-operative guidance and
quantitative assessment of therapeutic outcomes in robot-assisted Minimally
Invasive Surgery (MIS). 3D Deep Convolutional Neural Network (DCNN) is a viable
solution for this task, but is memory intensive. Small isotropic patches are
cropped from the original and large CT volume to mitigate this issue in
practice, but it may cause discontinuities between the adjacent patches and
severe class-imbalances within individual sub-volumes. This paper presents a
new 3D DCNN framework, namely Z-Net, to tackle the discontinuity and
class-imbalance issue by preserving a full field-of-view of the objects in the
XY planes using anisotropic spatial separable convolutions. The proposed Z-Net
can be seamlessly integrated into existing 3D DCNNs with isotropic convolutions
such as 3D U-Net and V-Net, with improved volume segmentation Intersection over
Union (IoU) - up to $12.6\%$. Detailed validation of Z-Net is provided for CT
aortic, liver and lung segmentation, demonstrating the effectiveness and
practical value of Z-Net for intra-operative 3D navigation in robot-assisted
MIS.
