  Deep reinforcement learning has achieved significant results in low-level
controlling tasks. However, for some applications like autonomous driving and
drone flying, it is difficult to control behavior stably since the agent may
suddenly change its actions which often lowers the controlling system's
efficiency, induces excessive mechanical wear, and causes uncontrollable,
dangerous behavior to the vehicle. Recently, a method called conditioning for
action policy smoothness (CAPS) was proposed to solve the problem of jerkiness
in low-dimensional features for applications such as quadrotor drones. To cope
with high-dimensional features, this paper proposes image-based regularization
for action smoothness (I-RAS) for solving jerky control in autonomous miniature
car racing. We also introduce a control based on impact ratio, an adaptive
regularization weight to control the smoothness constraint, called IR control.
In the experiment, an agent with I-RAS and IR control significantly improves
the success rate from 59% to 95%. In the real-world-track experiment, the agent
also outperforms other methods, namely reducing the average finish lap time,
while also improving the completion rate even without real world training. This
is also justified by an agent based on I-RAS winning the 2022 AWS DeepRacer
Final Championship Cup.
