  Model evaluation is of crucial importance in modern statistics application.
The construction of ROC and calculation of AUC have been widely used for binary
classification evaluation. Recent research generalizing the ROC/AUC analysis to
multi-class classification has problems in at least one of the four areas: 1.
failure to provide sensible plots 2. being sensitive to imbalanced data 3.
unable to specify mis-classification cost and 4. unable to provide evaluation
uncertainty quantification. Borrowing from a binomial matrix factorization
model, we provide an evaluation metric summarizing the pair-wise multi-class
True Positive Rate (TPR) and False Positive Rate (FPR) with one-dimensional
vector representation. Visualization on the representation vector measures the
relative speed of increment between TPR and FPR across all the classes pairs,
which in turns provides a ROC plot for the multi-class counterpart. An
integration over those factorized vector provides a binary AUC-equivalent
summary on the classifier performance. Mis-clasification weights specification
and bootstrapped confidence interval are also enabled to accommodate a variety
of of evaluation criteria. To support our findings, we conducted extensive
simulation studies and compared our method to the pair-wise averaged AUC
statistics on benchmark datasets.
