  Deep Reinforcement Learning (DRL) agents frequently face challenges in
adapting to tasks outside their training distribution, including issues with
over-fitting, catastrophic forgetting and sample inefficiency. Although the
application of adapters has proven effective in supervised learning contexts
such as natural language processing and computer vision, their potential within
the DRL domain remains largely unexplored. This paper delves into the
integration of adapters in reinforcement learning, presenting an innovative
adaptation strategy that demonstrates enhanced training efficiency and
improvement of the base-agent, experimentally in the nanoRTS environment, a
real-time strategy (RTS) game simulation. Our proposed universal approach is
not only compatible with pre-trained neural networks but also with rule-based
agents, offering a means to integrate human expertise.
