  In this article, we propose a novel probabilistic framework to improve the
accuracy of a weighted majority voting algorithm. In order to assign higher
weights to the classifiers which can correctly classify hard-to-classify
instances, we introduce the Item Response Theory (IRT) framework to evaluate
the samples' difficulty and classifiers' ability simultaneously. Three models
are created with different assumptions suitable for different cases. When
making an inference, we keep a balance between the accuracy and complexity. In
our experiment, all the base models are constructed by single trees via
bootstrap. To explain the models, we illustrate how the IRT ensemble model
constructs the classifying boundary. We also compare their performance with
other widely used methods and show that our model performs well on 19 datasets.
