  Due to lack of annotated pathological images, transfer learning has been the
predominant approach in the field of digital pathology.Pre-trained neural
networks based on ImageNet database are often used to extract "off the shelf"
features, achieving great success in predicting tissue types, molecular
features, and clinical outcomes, etc. We hypothesize that fine-tuning the
pre-trained models using histopathological images could further improve feature
extraction, and downstream prediction performance.We used 100,000 annotated HE
image patches for colorectal cancer (CRC) to finetune a pretrained Xception
model via a twostep approach.The features extracted from finetuned Xception
(FTX2048) model and Imagepretrained (IMGNET2048) model were compared through:
(1) tissue classification for HE images from CRC, same image type that was used
for finetuning; (2) prediction of immunerelated gene expression and (3) gene
mutations for lung adenocarcinoma (LUAD).Fivefold cross validation was used for
model performance evaluation. The extracted features from the finetuned FTX2048
exhibited significantly higher accuracy for predicting tisue types of CRC
compared to the off the shelf feature directly from Xception based on ImageNet
database. Particularly, FTX2048 markedly improved the accuracy for stroma from
87% to 94%. Similarly, features from FTX2048 boosted the prediction of
transcriptomic expression of immunerelated genesin LUAD. For the genes that had
signigicant relationships with image fetures, the features fgrom the finetuned
model imprroved the prediction for the majority of the genes. Inaddition,
fetures from FTX2048 improved prediction of mutation for 5 out of 9 most
frequently mutated genes in LUAD.
