  In real-world scenarios, the observation data for reinforcement learning with
continuous control is commonly noisy and part of it may be dynamically missing
over time, which violates the assumption of many current methods developed for
this. We addressed the issue within the framework of partially observable
Markov Decision Process (POMDP) using a model-based method, in which the
transition model is estimated from the incomplete and noisy observations using
a newly proposed surrogate loss function with local approximation, while the
policy and value function is learned with the help of belief imputation. For
the latter purpose, a generative model is constructed and is seamlessly
incorporated into the belief updating procedure of POMDP, which enables robust
execution even under a significant incompleteness and noise. The effectiveness
of the proposed method is verified on a collection of benchmark tasks, showing
that our approach outperforms several compared methods under various
challenging scenarios.
