  Cost-Sensitive Online Classification has drawn extensive attention in recent
years, where the main approach is to directly online optimize two well-known
cost-sensitive metrics: (i) weighted sum of sensitivity and specificity; (ii)
weighted misclassification cost. However, previous existing methods only
considered first-order information of data stream. It is insufficient in
practice, since many recent studies have proved that incorporating second-order
information enhances the prediction performance of classification models. Thus,
we propose a family of cost-sensitive online classification algorithms with
adaptive regularization in this paper. We theoretically analyze the proposed
algorithms and empirically validate their effectiveness and properties in
extensive experiments. Then, for better trade off between the performance and
efficiency, we further introduce the sketching technique into our algorithms,
which significantly accelerates the computational speed with quite slight
performance loss. Finally, we apply our algorithms to tackle several online
anomaly detection tasks from real world. Promising results prove that the
proposed algorithms are effective and efficient in solving cost-sensitive
online classification problems in various real-world domains.
