  Accurate prediction of user consumption is a key part not only in
understanding consumer flexibility and behavior patterns, but in the design of
robust and efficient energy saving programs as well. Existing prediction
methods usually have high relative errors that can be larger than 30% and have
difficulties accounting for heterogeneity between individual users. In this
paper, we propose a method to improve prediction accuracy of individual users
by adaptively exploring sparsity in historical data and leveraging predictive
relationship between different users. Sparsity is captured by popular least
absolute shrinkage and selection estimator, while user selection is formulated
as an optimal hypothesis testing problem and solved via a covariance test.
Using real world data from PG&E, we provide extensive simulation validation of
the proposed method against well-known techniques such as support vector
machine, principle component analysis combined with linear regression, and
random forest. The results demonstrate that our proposed methods are
operationally efficient because of linear nature, and achieve optimal
prediction performance.
