  Due to physiological variation, patients diagnosed with the same condition
may exhibit divergent, but related, responses to the same treatments. Hidden
Parameter Markov Decision Processes (HiP-MDPs) tackle this transfer-learning
problem by embedding these tasks into a low-dimensional space. However, the
original formulation of HiP-MDP had a critical flaw: the embedding uncertainty
was modeled independently of the agent's state uncertainty, requiring an
unnatural training procedure in which all tasks visited every part of the state
space---possible for robots that can be moved to a particular location,
impossible for human patients. We update the HiP-MDP framework and extend it to
more robustly develop personalized medicine strategies for HIV treatment.
