  Pancreatic cancer is one of the most malignant cancers in the world, which
deteriorates rapidly with very high mortality. The rapid on-site evaluation
(ROSE) technique innovates the workflow by immediately analyzing the fast
stained cytopathological images with on-site pathologists, which enables faster
diagnosis in this time-pressured process. However, the wider expansion of ROSE
diagnosis has been hindered by the lack of experienced pathologists. To
overcome this problem, we propose a hybrid high-performance deep learning model
to enable the automated workflow, thus freeing the occupation of the valuable
time of pathologists. By firstly introducing the Transformer block into this
field with our particular multi-stage hybrid design, the spatial features
generated by the convolutional neural network (CNN) significantly enhance the
Transformer global modeling. Turning multi-stage spatial features as global
attention guidance, this design combines the robustness from the inductive bias
of CNN with the sophisticated global modeling power of Transformer. A dataset
of 4240 ROSE images is collected to evaluate the method in this unexplored
field. The proposed multi-stage hybrid Transformer (MSHT) achieves 95.68% in
classification accuracy, which is distinctively higher than the
state-of-the-art models. Facing the need for interpretability, MSHT outperforms
its counterparts with more accurate attention regions. The results demonstrate
that the MSHT can distinguish cancer samples accurately at an unprecedented
image scale, laying the foundation for deploying automatic decision systems and
enabling the expansion of ROSE in clinical practice. The code and records are
available at: https://github.com/sagizty/Multi-Stage-Hybrid-Transformer.
