  Recent advances in Neural Architecture Search (NAS) have produced
state-of-the-art architectures on several tasks. NAS shifts the efforts of
human experts from developing novel architectures directly to designing
architecture search spaces and methods to explore them efficiently. The search
space definition captures prior knowledge about the properties of the
architectures and it is crucial for the complexity and the performance of the
search algorithm. However, different search space definitions require
restarting the learning process from scratch. We propose a novel agent based on
the Transformer that supports joint training and efficient transfer of prior
knowledge between multiple search spaces and tasks.
