  In this work we investigate an importance sampling approach for evaluating
policies for a structurally time-varying factored Markov decision process
(MDP), i.e. the policy's value is estimated with a high-probability confidence
interval. In particular, we begin with a multi-agent MDP controlled by a known
policy but with unknown transition dynamics. One agent is then removed from the
system - i.e. the system experiences node dropout - forming a new MDP of the
remaining agents, with a new state space, action space, and new transition
dynamics. We assume that the effect of removing an agent corresponds to the
marginalization of its factor in the transition dynamics. The reward function
may likewise be marginalized, or it may be entirely redefined for the new
system. Robust policy importance sampling is then used to evaluate candidate
policies for the new system, and estimated values are presented with
probabilistic confidence bounds. This computation is completed with no
observations of the new system, meaning that a safe policy may be found before
dropout occurs. The utility of this approach is demonstrated in simulation and
compared to Monte Carlo simulation of the new system.
