  Echocardiography is essential to modern cardiology. However, human
interpretation limits high throughput analysis, limiting echocardiography from
reaching its full clinical and research potential for precision medicine. Deep
learning is a cutting-edge machine-learning technique that has been useful in
analyzing medical images but has not yet been widely applied to
echocardiography, partly due to the complexity of echocardiograms' multi view,
multi modality format. The essential first step toward comprehensive computer
assisted echocardiographic interpretation is determining whether computers can
learn to recognize standard views. To this end, we anonymized 834,267
transthoracic echocardiogram (TTE) images from 267 patients (20 to 96 years, 51
percent female, 26 percent obese) seen between 2000 and 2017 and labeled them
according to standard views. Images covered a range of real world clinical
variation. We built a multilayer convolutional neural network and used
supervised learning to simultaneously classify 15 standard views. Eighty
percent of data used was randomly chosen for training and 20 percent reserved
for validation and testing on never seen echocardiograms. Using multiple images
from each clip, the model classified among 12 video views with 97.8 percent
overall test accuracy without overfitting. Even on single low resolution
images, test accuracy among 15 views was 91.7 percent versus 70.2 to 83.5
percent for board-certified echocardiographers. Confusional matrices, occlusion
experiments, and saliency mapping showed that the model finds recognizable
similarities among related views and classifies using clinically relevant image
features. In conclusion, deep neural networks can classify essential
echocardiographic views simultaneously and with high accuracy. Our results
provide a foundation for more complex deep learning assisted echocardiographic
interpretation.
