  Cannulation for hemodialysis is the act of inserting a needle into a
surgically created vascular access (e.g., an arteriovenous fistula) for the
purpose of dialysis. The main risk associated with cannulation is infiltration,
the puncture of the wall of the vascular access after entry, which can cause
medical complications. Simulator-based training allows clinicians to gain
cannulation experience without putting patients at risk. In this paper, we
propose to use deep-learning-based techniques for detecting, based on video,
whether the needle tip is in or has infiltrated the simulated fistula. Three
categories of deep neural networks are investigated in this work: modified
pre-trained models based on VGG-16 and ResNet-50, light convolutional neural
networks (light CNNs), and convolutional recurrent neural networks (CRNNs).
CRNNs consist of convolutional layers and a long short-term memory (LSTM)
layer. A data set of cannulation experiments was collected and analyzed. The
results show that both the light CNN and the CRNN achieve better performance
than the pre-trained baseline models. The CRNN was implemented in real time on
commodity hardware for use in the cannulation simulator, and the performance
was verified. Deep-learning video analysis is a viable method for detecting
needle state in a low cost cannulation simulator. Our data sets and code are
released at https://github.com/axin233/DL_for_Needle_Detection_Cannulation
