  This paper presents a technique to train a robot to perform kick-motion in AI
soccer by using reinforcement learning (RL). In RL, an agent interacts with an
environment and learns to choose an action in a state at each step. When
training RL algorithms, a problem called the curse of dimensionality (COD) can
occur if the dimension of the state is high and the number of training data is
low. The COD often causes degraded performance of RL models. In the situation
of the robot kicking the ball, as the ball approaches the robot, the robot
chooses the action based on the information obtained from the soccer field. In
order not to suffer COD, the training data, which are experiences in the case
of RL, should be collected evenly from all areas of the soccer field over
(theoretically infinite) time. In this paper, we attempt to use the relative
coordinate system (RCS) as the state for training kick-motion of robot agent,
instead of using the absolute coordinate system (ACS). Using the RCS eliminates
the necessity for the agent to know all the (state) information of entire
soccer field and reduces the dimension of the state that the agent needs to
know to perform kick-motion, and consequently alleviates COD. The training
based on the RCS is performed with the widely used Deep Q-network (DQN) and
tested in the AI Soccer environment implemented with Webots simulation
software.
