  Swimming microrobots are increasingly developed with complex materials and
dynamic shapes and are expected to operate in complex environments in which the
system dynamics are difficult to model and positional control of the microrobot
is not straightforward to achieve. Deep reinforcement learning is a promising
method of autonomously developing robust controllers for creating smart
microrobots, which can adapt their behavior to operate in uncharacterized
environments without the need to model the system dynamics. Here, we report the
development of a smart helical magnetic hydrogel microrobot that used the soft
actor critic reinforcement learning algorithm to autonomously derive a control
policy which allowed the microrobot to swim through an uncharacterized
biomimetic fluidic environment under control of a time varying magnetic field
generated from a three-axis array of electromagnets. The reinforcement learning
agent learned successful control policies with fewer than 100,000 training
steps, demonstrating sample efficiency for fast learning. We also demonstrate
that we can fine tune the control policies learned by the reinforcement
learning agent by fitting mathematical functions to the learned policy's action
distribution via regression. Deep reinforcement learning applied to microrobot
control is likely to significantly expand the capabilities of the next
generation of microrobots.
