  Behavioural cloning uses a dataset of demonstrations to learn a behavioural
policy. To overcome various learning and policy adaptation problems, we propose
to use latent space to index a demonstration dataset, instantly access similar
relevant experiences, and copy behavior from these situations. Actions from a
selected similar situation can be performed by the agent until representations
of the agent's current situation and the selected experience diverge in the
latent space. Thus, we formulate our control problem as a search problem over a
dataset of experts' demonstrations. We test our approach on BASALT
MineRL-dataset in the latent representation of a Video PreTraining model. We
compare our model to state-of-the-art Minecraft agents. Our approach can
effectively recover meaningful demonstrations and show human-like behavior of
an agent in the Minecraft environment in a wide variety of scenarios.
Experimental results reveal that performance of our search-based approach is
comparable to trained models, while allowing zero-shot task adaptation by
changing the demonstration examples.
