  Background: Single-cell RNA sequencing (scRNA-seq) yields valuable insights
about gene expression and gives critical information about complex tissue
cellular composition. In the analysis of single-cell RNA sequencing, the
annotations of cell subtypes are often done manually, which is time-consuming
and irreproducible. Garnett is a cell-type annotation software based the on
elastic net method. Besides cell-type annotation, supervised machine learning
methods can also be applied to predict other cell phenotypes from genomic data.
Despite the popularity of such applications, there is no existing study to
systematically investigate the performance of those supervised algorithms in
various sizes of scRNA-seq data sets.
  Methods and Results: This study evaluates 13 popular supervised machine
learning algorithms to classify cell phenotypes, using published real and
simulated data sets with diverse cell sizes. The benchmark contained two parts.
In the first part, we used real data sets to assess the popular supervised
algorithms' computing speed and cell phenotype classification performance. The
classification performances were evaluated using AUC statistics, F1-score,
precision, recall, and false-positive rate. In the second part, we evaluated
gene selection performance using published simulated data sets with a known
list of real genes.
  Conclusion: The study outcomes showed that ElasticNet with interactions
performed best in small and medium data sets. NB was another appropriate method
for medium data sets. In large data sets, XGB works excellent. Ensemble
algorithms were not significantly superior to individual machine learning
methods. Adding interactions to ElasticNet can help, and the improvement was
significant in small data sets.
