  Specialized domain knowledge is often necessary to accurately annotate
training sets for in-depth analysis, but can be burdensome and time-consuming
to acquire from domain experts. This issue arises prominently in automated
behavior analysis, in which agent movements or actions of interest are detected
from video tracking data. To reduce annotation effort, we present TREBA: a
method to learn annotation-sample efficient trajectory embedding for behavior
analysis, based on multi-task self-supervised learning. The tasks in our method
can be efficiently engineered by domain experts through a process we call "task
programming", which uses programs to explicitly encode structured knowledge
from domain experts. Total domain expert effort can be reduced by exchanging
data annotation time for the construction of a small number of programmed
tasks. We evaluate this trade-off using data from behavioral neuroscience, in
which specialized domain knowledge is used to identify behaviors. We present
experimental results in three datasets across two domains: mice and fruit
flies. Using embeddings from TREBA, we reduce annotation burden by up to a
factor of 10 without compromising accuracy compared to state-of-the-art
features. Our results thus suggest that task programming and self-supervision
can be an effective way to reduce annotation effort for domain experts.
