  The surgical operating room (OR) presents many opportunities for automation
and optimization. Videos from various sources in the OR are becoming
increasingly available. The medical community seeks to leverage this wealth of
data to develop automated methods to advance interventional care, lower costs,
and improve overall patient outcomes. Existing datasets from OR room cameras
are thus far limited in size or modalities acquired, leaving it unclear which
sensor modalities are best suited for tasks such as recognizing surgical action
from videos. This study demonstrates that surgical action recognition
performance can vary depending on the image modalities used. We perform a
methodical analysis on several commonly available sensor modalities, presenting
two fusion approaches that improve classification performance. The analyses are
carried out on a set of multi-view RGB-D video recordings of 18 laparoscopic
procedures.
