  Neutronic calculations for reactors are a daunting task when using Monte
Carlo (MC) methods. As high-performance computing has advanced, the simulation
of a reactor is nowadays more readily done, but design and optimization with
multiple parameters is still a computational challenge. MC transport
simulations, coupled with machine learning techniques, offer promising avenues
for enhancing the efficiency and effectiveness of nuclear reactor optimization.
This paper introduces a novel benchmark problem within the OpenNeoMC framework
designed specifically for reinforcement learning. The benchmark involves
optimizing a unit cell of a research reactor with two varying parameters (fuel
density and water spacing) to maximize neutron flux while maintaining reactor
criticality. The test case features distinct local optima, representing
different physical regimes, thus posing a challenge for learning algorithms.
Through extensive simulations utilizing evolutionary and neuroevolutionary
algorithms, we demonstrate the effectiveness of reinforcement learning in
navigating complex optimization landscapes with strict constraints.
Furthermore, we propose acceleration techniques within the OpenNeoMC framework,
including model updating and cross-section usage by RAM utilization, to
expedite simulation times. Our findings emphasize the importance of machine
learning integration in reactor optimization and contribute to advancing
methodologies for addressing intricate optimization challenges in nuclear
engineering. The sources of this work are available at our GitHub repository:
https://github.com/Scientific-Computing-Lab-NRCN/RLOpenNeoMC
