  Herein, we introduce "Look and Diagnose" (LAD), a hybrid deep learning-based
system that aims to support doctors in the medical field in diagnosing
effectively the Benign Paroxysmal Positional Vertigo (BPPV) disorder. Given the
body postures of the patient in the Dix-Hallpike and lateral head turns test,
the visual information of both eyes is captured and fed into LAD for analyzing
and classifying into one of six possible disorders the patient might be
suffering from. The proposed system consists of two streams: (1) an RNN-based
stream that takes raw RGB images of both eyes to extract visual features and
optical flow of each eye followed by ternary classification to determine
left/right posterior canal (PC) or other; and (2) pupil detector stream that
detects the pupil when it is classified as Non-PC and classifies the direction
and strength of the beating to categorize the Non-PC types into the remaining
four classes: Geotropic BPPV (left and right) and Apogeotropic BPPV (left and
right). Experimental results show that with the patient's body postures, the
system can accurately classify given BPPV disorder into the six types of
disorders with an accuracy of 91% on the validation set. The proposed method
can successfully classify disorders with an accuracy of 93% for the Posterior
Canal disorder and 95% for the Geotropic and Apogeotropic disorder, paving a
potential direction for research with the medical data.
