  Complex video queries can be answered by decomposing them into modular
subtasks. However, existing video data management systems assume the existence
of predefined modules for each subtask. We introduce VOCAL-UDF, a novel
self-enhancing system that supports compositional queries over videos without
the need for predefined modules. VOCAL-UDF automatically identifies and
constructs missing modules and encapsulates them as user-defined functions
(UDFs), thus expanding its querying capabilities. To achieve this, we formulate
a unified UDF model that leverages large language models (LLMs) to aid in new
UDF generation. VOCAL-UDF handles a wide range of concepts by supporting both
program-based UDFs (i.e., Python functions generated by LLMs) and
distilled-model UDFs (lightweight vision models distilled from strong
pretrained models). To resolve the inherent ambiguity in user intent, VOCAL-UDF
generates multiple candidate UDFs and uses active learning to efficiently
select the best one. With the self-enhancing capability, VOCAL-UDF
significantly improves query performance across three video datasets.
