  This paper introduces ViT4LPA, an innovative Vision Transformer (ViT) based
approach for Load Profile Analysis (LPA). We transform time-series load
profiles into load images. This allows us to leverage the ViT architecture,
originally designed for image processing, as a pre-trained image encoder to
uncover latent patterns within load data. ViT is pre-trained using an extensive
load image dataset, comprising 1M load images derived from smart meter data
collected over a two-year period from 2,000 residential users. The training
methodology is self-supervised, masked image modeling, wherein masked load
images are restored to reveal hidden relationships among image patches. The
pre-trained ViT encoder is then applied to various downstream tasks, including
the identification of electric vehicle (EV) charging loads and behind-the-meter
solar photovoltaic (PV) systems and load disaggregation. Simulation results
illustrate ViT4LPA's superior performance compared to existing neural network
models in downstream tasks. Additionally, we conduct an in-depth analysis of
the attention weights within the ViT4LPA model to gain insights into its
information flow mechanisms.
