  Microprocessor architects are increasingly resorting to domain-specific
customization in the quest for high-performance and energy-efficiency. As the
systems grow in complexity, fine-tuning architectural parameters across
multiple sub-systems (e.g., datapath, memory blocks in different hierarchies,
interconnects, compiler optimization, etc.) quickly results in a combinatorial
explosion of design space. This makes domain-specific customization an
extremely challenging task. Prior work explores using reinforcement learning
(RL) and other optimization methods to automatically explore the large design
space. However, these methods have traditionally relied on single-agent RL/ML
formulations. It is unclear how scalable single-agent formulations are as we
increase the complexity of the design space (e.g., full stack System-on-Chip
design). Therefore, we propose an alternative formulation that leverages
Multi-Agent RL (MARL) to tackle this problem. The key idea behind using MARL is
an observation that parameters across different sub-systems are more or less
independent, thus allowing a decentralized role assigned to each agent. We test
this hypothesis by designing domain-specific DRAM memory controller for several
workload traces. Our evaluation shows that the MARL formulation consistently
outperforms single-agent RL baselines such as Proximal Policy Optimization and
Soft Actor-Critic over different target objectives such as low power and
latency. To this end, this work opens the pathway for new and promising
research in MARL solutions for hardware architecture search.
