  A multi-modal machine learning system uses multiple unique data sources and
types to improve its performance. This article proposes a system that combines
results from several types of models, all of which are trained on different
data signals. As an example to illustrate the efficacy of the system, an
experiment is described in which multiple types of data are collected from rats
suffering from seizures. This data includes electrocorticography readings,
piezoelectric motion sensor data, and video recordings. Separate models are
trained on each type of data, with the goal of classifying each time frame as
either containing a seizure or not. After each model has generated its
classification predictions, these results are combined. While each data signal
works adequately on its own for prediction purposes, the significant imbalance
in class labels leads to increased numbers of false positives, which can be
filtered and removed by utilizing all data sources. This paper will demonstrate
that, after postprocessing and combination techniques, classification accuracy
is improved with this multi-modal system when compared to the performance of
each individual data source.
