  Reinforcement learning (RL) is a subset of artificial intelligence (AI) where
agents learn the best action by interacting with the environment, making it
suitable for tasks that do not require labeled data or direct supervision.
Hyperparameters (HP) tuning refers to choosing the best parameter that leads to
optimal solutions in RL algorithms. Manual or random tuning of the HP may be a
crucial process because variations in this parameter lead to changes in the
overall learning aspects and different rewards. In this paper, a novel and
automatic HP-tuning method called Q-FOX is proposed. This uses both the FOX
optimizer, a new optimization method inspired by nature that mimics red foxes'
hunting behavior, and the commonly used, easy-to-implement RL Q-learning
algorithm to solve the problem of HP tuning. Moreover, a new objective function
is proposed which prioritizes the reward over the mean squared error (MSE) and
learning time (steps). Q-FOX has been evaluated on two OpenAI Gym environment
control tasks: Cart Pole and Frozen Lake. It exposed greater cumulative rewards
than HP tuning with other optimizers, such as PSO, GA, Bee, or randomly
selected HP. The cumulative reward for the Cart Pole task was 32.08, and for
the Frozen Lake task was 0.95. Despite the robustness of Q-FOX, it has
limitations. It cannot be used directly in real-word problems before choosing
the HP in a simulation environment because its processes work iteratively,
making it time-consuming. The results indicate that Q-FOX has played an
essential role in HP tuning for RL algorithms to effectively solve different
control tasks.
