  To predict lung nodule malignancy with a high sensitivity and specificity, we
propose a fusion algorithm that combines handcrafted features (HF) into the
features learned at the output layer of a 3D deep convolutional neural network
(CNN). First, we extracted twenty-nine handcrafted features, including nine
intensity features, eight geometric features, and twelve texture features based
on grey-level co-occurrence matrix (GLCM) averaged from thirteen directions. We
then trained 3D CNNs modified from three state-of-the-art 2D CNN architectures
(AlexNet, VGG-16 Net and Multi-crop Net) to extract the CNN features learned at
the output layer. For each 3D CNN, the CNN features combined with the 29
handcrafted features were used as the input for the support vector machine
(SVM) coupled with the sequential forward feature selection (SFS) method to
select the optimal feature subset and construct the classifiers. The fusion
algorithm takes full advantage of the handcrafted features and the highest
level CNN features learned at the output layer. It can overcome the
disadvantage of the handcrafted features that may not fully reflect the unique
characteristics of a particular lesion by combining the intrinsic CNN features.
Meanwhile, it also alleviates the requirement of a large scale annotated
dataset for the CNNs based on the complementary of handcrafted features. The
patient cohort includes 431 malignant nodules and 795 benign nodules extracted
from the LIDC/IDRI database. For each investigated CNN architecture, the
proposed fusion algorithm achieved the highest AUC, accuracy, sensitivity, and
specificity scores among all competitive classification models.
