  In analyses of rare-events, regardless of the domain of application,
class-imbalance issue is intrinsic. Although the challenges are known to data
experts, their explicit impact on the analytic and the decisions made based on
the findings are often overlooked. This is in particular prevalent in
interdisciplinary research where the theoretical aspects are sometimes
overshadowed by the challenges of the application. To show-case these
undesirable impacts, we conduct a series of experiments on a recently created
benchmark data, named Space Weather ANalytics for Solar Flares (SWAN-SF). This
is a multivariate time series dataset of magnetic parameters of active regions.
As a remedy for the imbalance issue, we study the impact of data manipulation
(undersampling and oversampling) and model manipulation (using class weights).
Furthermore, we bring to focus the auto-correlation of time series that is
inherited from the use of sliding window for monitoring flares' history.
Temporal coherence, as we call this phenomenon, invalidates the randomness
assumption, thus impacting all sampling practices including different
cross-validation techniques. We illustrate how failing to notice this concept
could give an artificial boost in the forecast performance and result in
misleading findings. Throughout this study we utilized Support Vector Machine
as a classifier, and True Skill Statistics as a verification metric for
comparison of experiments. We conclude our work by specifying the correct
practice in each case, and we hope that this study could benefit researchers in
other domains where time series of rare events are of interest.
