  Incorporating the successful paradigm of pretraining and finetuning from
Computer Vision and Natural Language Processing into decision-making has become
increasingly popular in recent years. In this paper, we study Imitation
Learning from Observation with pretrained models and find existing approaches
such as BCO and AIME face knowledge barriers, specifically the Embodiment
Knowledge Barrier (EKB) and the Demonstration Knowledge Barrier (DKB), greatly
limiting their performance. The EKB arises when pretrained models lack
knowledge about unseen observations, leading to errors in action inference. The
DKB results from policies trained on limited demonstrations, hindering
adaptability to diverse scenarios. We thoroughly analyse the underlying
mechanism of these barriers and propose AIME-v2 upon AIME as a solution.
AIME-v2 uses online interactions with data-driven regulariser to alleviate the
EKB and mitigates the DKB by introducing a surrogate reward function to enhance
policy training. Experimental results on tasks from the DeepMind Control Suite
and Meta-World benchmarks demonstrate the effectiveness of these modifications
in improving both sample-efficiency and converged performance. The study
contributes valuable insights into resolving knowledge barriers for enhanced
decision-making in pretraining-based approaches. Code will be available at
https://github.com/argmax-ai/aime-v2.
