  With the rapid growth of renewable energy, lots of small photovoltaic (PV)
prosumers emerge. Due to the uncertainty of solar power generation, there is a
need for aggregated prosumers to predict solar power generation and whether
solar power generation will be larger than load. This paper presents two
interpretable neural networks to solve the problem: one binary classification
neural network and one regression neural network. The neural networks are built
using TensorFlow. The global feature importance and local feature contributions
are examined by three gradient-based methods: Integrated Gradients, Expected
Gradients, and DeepLIFT. Moreover, we detect abnormal cases when predictions
might fail by estimating the prediction uncertainty using Bayesian neural
networks. Neural networks, which are interpreted by gradient-based methods and
complemented with uncertainty estimation, provide robust and explainable
forecasting for decision-makers.
