  Keeping the balance between electricity generation and consumption is
becoming increasingly challenging and costly, mainly due to the rising share of
renewables, electric vehicles and heat pumps and electrification of industrial
processes. Accurate imbalance forecasts, along with reliable uncertainty
estimations, enable transmission system operators (TSOs) to dispatch
appropriate reserve volumes, reducing balancing costs. Further, market parties
can use these probabilistic forecasts to design strategies that exploit asset
flexibility to help balance the grid, generating revenue with known risks.
Despite its importance, literature regarding system imbalance (SI) forecasting
is limited. Further, existing methods do not focus on situations with high
imbalance magnitude, which are crucial to forecast accurately for both TSOs and
market parties. Hence, we propose an ensemble of C-VSNs, which are our
adaptation of variable selection networks (VSNs). Each minute, our model
predicts the imbalance of the current and upcoming two quarter-hours, along
with uncertainty estimations on these forecasts. We evaluate our approach by
forecasting the imbalance of Belgium, where high imbalance magnitude is defined
as $|$SI$| > 500\,$MW (occurs 1.3% of the time in Belgium). For high imbalance
magnitude situations, our model outperforms the state-of-the-art by 23.4% (in
terms of continuous ranked probability score (CRPS), which evaluates
probabilistic forecasts), while also attaining a 6.5% improvement in overall
CRPS. Similar improvements are achieved in terms of root-mean-squared error.
Additionally, we developed a fine-tuning methodology to effectively include new
inputs with limited history in our model. This work was performed in
collaboration with Elia (the Belgian TSO) to further improve their imbalance
forecasts, demonstrating the relevance of our work.
