  The traditional k nearest neighbor (kNN) approach uses a distance formula
within a spherical region to determine the k closest training observations to a
test sample point. However, this approach may not work well when test point is
located outside this region. Moreover, aggregating many base kNN learners can
result in poor ensemble performance due to high classification errors. To
address these issues, a new optimal extended neighborhood rule based ensemble
method is proposed in this paper. This rule determines neighbors in k steps
starting from the closest sample point to the unseen observation and selecting
subsequent nearest data points until the required number of observations is
reached. Each base model is constructed on a bootstrap sample with a random
subset of features, and optimal models are selected based on out-of-bag
performance after building a sufficient number of models. The proposed ensemble
is compared with state-of-the-art methods on 17 benchmark datasets using
accuracy, Cohen's kappa, and Brier score (BS). The performance of the proposed
method is also assessed by adding contrived features in the original data.
