  Recent work has shown the promise of creating generalist, transformer-based,
models for language, vision, and sequential decision-making problems. To create
such models, we generally require centralized training objectives, data, and
compute. It is of interest if we can more flexibly create generalist policies
by merging together multiple, task-specific, individually trained policies. In
this work, we take a preliminary step in this direction through merging, or
averaging, subsets of Decision Transformers in parameter space trained on
different MuJoCo locomotion problems, forming multi-task models without
centralized training. We also demonstrate the importance of various
methodological choices when merging policies, such as utilizing common
pre-trained initializations, increasing model capacity, and utilizing Fisher
information for weighting parameter importance. In general, we believe research
in this direction could help democratize and distribute the process that forms
multi-task robotics policies. Our implementation is available at
https://github.com/daniellawson9999/merging-decision-transformers.
