  Adaptive brain stimulation can treat neurological conditions such as
Parkinson's disease and post-stroke motor deficits by influencing abnormal
neural activity. Because of patient heterogeneity, each patient requires a
unique stimulation policy to achieve optimal neural responses. Model-free
reinforcement learning (MFRL) holds promise in learning effective policies for
a variety of similar control tasks, but is limited in domains like brain
stimulation by a need for numerous costly environment interactions. In this
work we introduce Coprocessor Actor Critic, a novel, model-based reinforcement
learning (MBRL) approach for learning neural coprocessor policies for brain
stimulation. Our key insight is that coprocessor policy learning is a
combination of learning how to act optimally in the world and learning how to
induce optimal actions in the world through stimulation of an injured brain. We
show that our approach overcomes the limitations of traditional MFRL methods in
terms of sample efficiency and task success and outperforms baseline MBRL
approaches in a neurologically realistic model of an injured brain.
