  The Hardware Trojan (HT) problem can be thought of as a continuous game
between attackers and defenders, each striving to outsmart the other by
leveraging any available means for an advantage. Machine Learning (ML) has
recently been key in advancing HT research. Various novel techniques, such as
Reinforcement Learning (RL) and Graph Neural Networks (GNNs), have shown HT
insertion and detection capabilities. HT insertion with ML techniques,
specifically, has seen a spike in research activity due to the shortcomings of
conventional HT benchmarks and the inherent human design bias that occurs when
we create them. This work continues this innovation by presenting a tool called
"TrojanForge", capable of generating HT adversarial examples that defeat HT
detectors; demonstrating the capabilities of GAN-like adversarial tools for
automatic HT insertion. We introduce an RL environment where the RL insertion
agent interacts with HT detectors in an insertion-detection loop where the
agent collects rewards based on its success in bypassing HT detectors. Our
results show that this process leads to inserted HTs that evade various HT
detectors, achieving high attack success percentages. This tool provides
insight into why HT insertion fails in some instances and how we can leverage
this knowledge in defense.
