  Functional Electrical Stimulation (FES) is a technique to evoke muscle
contraction through low-energy electrical signals. FES can animate paralysed
limbs. Yet, an open challenge remains on how to apply FES to achieve desired
movements. This challenge is accentuated by the complexities of human bodies
and the non-stationarities of the muscles' responses. The former causes
difficulties in performing inverse dynamics, and the latter causes control
performance to degrade over extended periods of use. Here, we engage the
challenge via a data-driven approach. Specifically, we learn to control FES
through Reinforcement Learning (RL) which can automatically customise the
stimulation for the patients. However, RL typically has Markovian assumptions
while FES control systems are non-Markovian because of the non-stationarities.
To deal with this problem, we use a recurrent neural network to create
Markovian state representations. We cast FES controls into RL problems and
train RL agents to control FES in different settings in both simulations and
the real world. The results show that our RL controllers can maintain control
performances over long periods and have better stimulation characteristics than
PID controllers.
