  In recent years, deep reinforcement learning has achieved significant results
in low-level controlling tasks. However, the problem of control smoothness has
less attention. In autonomous driving, unstable control is inevitable since the
vehicle might suddenly change its actions. This problem will lower the
controlling system's efficiency, induces excessive mechanical wear, and causes
uncontrollable, dangerous behavior to the vehicle. In this paper, we apply the
Conditioning for Action Policy Smoothness (CAPS) with image-based input to
smooth the control of an autonomous miniature car racing. Applying CAPS and
sim-to-real transfer methods helps to stabilize the control at a higher speed.
Especially, the agent with CAPS and CycleGAN reduces 21.80% of the average
finishing lap time. Moreover, we also conduct extensive experiments to analyze
the impact of CAPS components.
