  Chaos-based reinforcement learning (CBRL) is a method in which the agent's
internal chaotic dynamics drives exploration. This approach offers a model for
considering how the biological brain can create variability in its behavior and
learn in an exploratory manner. At the same time, it is a learning model that
has the ability to automatically switch between exploration and exploitation
modes and the potential to realize higher explorations that reflect what it has
learned so far. However, the learning algorithms in CBRL have not been
well-established in previous studies and have yet to incorporate recent
advances in reinforcement learning. This study introduced Twin Delayed Deep
Deterministic Policy Gradients (TD3), which is one of the state-of-the-art deep
reinforcement learning algorithms that can treat deterministic and continuous
action spaces, to CBRL. The validation results provide several insights. First,
TD3 works as a learning algorithm for CBRL in a simple goal-reaching task.
Second, CBRL agents with TD3 can autonomously suppress their exploratory
behavior as learning progresses and resume exploration when the environment
changes. Finally, examining the effect of the agent's chaoticity on learning
shows that extremely strong chaos negatively impacts the flexible switching
between exploration and exploitation.
