  Data subsampling has become widely recognized as a tool to overcome
computational and economic bottlenecks in analyzing massive datasets. We
contribute to the development of adaptive design for estimation of finite
population characteristics, using active learning and adaptive importance
sampling. We propose an active sampling strategy that iterates between
estimation and data collection with optimal subsamples, guided by machine
learning predictions on yet unseen data. The method is illustrated on virtual
simulation-based safety assessment of advanced driver assistance systems.
Substantial performance improvements are demonstrated compared to traditional
sampling methods.
