  A standard ML model is commonly generated by a single method that specifies
aspects such as architecture, initialization, training data and hyperparameters
configuration. The presented work introduces a novel methodology allowing to
define multiple methods as distinct agents. Agents can collaborate and compete
to generate and improve ML models for a given tasks. The proposed methodology
is demonstrated with the generation and extension of a dynamic modular
multitask ML system solving more than one hundred image classification tasks.
Diverse agents can compete to produce the best performing model for a task by
reusing the modules introduced to the system by competing agents. The presented
work focuses on the study of agents capable of: 1) reusing the modules
generated by concurrent agents, 2) activating in parallel multiple modules in a
frozen state by connecting them with trainable modules, 3) condition the
activation mixture on each data sample by using a trainable router module. We
demonstrate that this simple per-sample parallel routing method can boost the
quality of the combined solutions by training a fraction of the activated
parameters.
