  This study explores the crucial interplay between aggregators and building
occupants in activating flexibility through Demand Response (DR) programs, with
a keen focus on achieving robust decarbonization and fortifying the resilience
of the energy system amidst the uncertainties presented by Renewable Energy
Sources (RES). Firstly, it introduces a methodology of optimizing aggregated
flexibility provision strategies in environments with limited data, utilizing
Discrete Fourier Transformation (DFT) and clustering techniques to identify
building occupant's activity patterns. Secondly, the study assesses the
disaggregated flexibility provision of Heating Ventilation and Air Conditioning
(HVAC) systems during DR events, employing machine learning and optimization
techniques for precise, device-level analysis. The first approach offers a
non-intrusive pathway for aggregators to provide flexibility services in
environments of a single smart meter for the whole building's consumption,
while the second approach carefully considers building occupants' thermal
comfort profiles, while maximizing flexibility in case of existence of
dedicated smart meters to the HVAC systems. Through the application of
data-driven techniques and encompassing case studies from both industrial and
residential buildings, this paper not only unveils pivotal opportunities for
aggregators in the balancing and emerging flexibility markets but also
successfully develops end-to-end practical tools for aggregators. Furthermore,
the efficacy of this tool is validated through detailed case studies,
substantiating its operational capability and contributing to the evolution of
a resilient and efficient energy system.
