  We propose a novel point cloud based 3D organ segmentation pipeline utilizing
deep Q-learning. In order to preserve shape properties, the learning process is
guided using a statistical shape model. The trained agent directly predicts
piece-wise linear transformations for all vertices in each iteration. This
mapping between the ideal transformation for an object outline estimation is
learned based on image features. To this end, we introduce aperture features
that extract gray values by sampling the 3D volume within the cone centered
around the associated vertex and its normal vector. Our approach is also
capable of estimating a hierarchical pyramid of non rigid deformations for
multi-resolution meshes. In the application phase, we use a marginal approach
to gradually estimate affine as well as non-rigid transformations. We performed
extensive evaluations to highlight the robust performance of our approach on a
variety of challenge data as well as clinical data. Additionally, our method
has a run time ranging from 0.3 to 2.7 seconds to segment each organ. In
addition, we show that the proposed method can be applied to different organs,
X-ray based modalities, and scanning protocols without the need of transfer
learning. As we learn actions, even unseen reference meshes can be processed as
demonstrated in an example with the Visible Human. From this we conclude that
our method is robust, and we believe that our method can be successfully
applied to many more applications, in particular, in the interventional imaging
space.
