  In this paper, we propose a policy gradient method for confounded partially
observable Markov decision processes (POMDPs) with continuous state and
observation spaces in the offline setting. We first establish a novel
identification result to non-parametrically estimate any history-dependent
policy gradient under POMDPs using the offline data. The identification enables
us to solve a sequence of conditional moment restrictions and adopt the min-max
learning procedure with general function approximation for estimating the
policy gradient. We then provide a finite-sample non-asymptotic bound for
estimating the gradient uniformly over a pre-specified policy class in terms of
the sample size, length of horizon, concentratability coefficient and the
measure of ill-posedness in solving the conditional moment restrictions.
Lastly, by deploying the proposed gradient estimation in the gradient ascent
algorithm, we show the global convergence of the proposed algorithm in finding
the history-dependent optimal policy under some technical conditions. To the
best of our knowledge, this is the first work studying the policy gradient
method for POMDPs under the offline setting.
