  Partially Observable Markov Decision Process (POMDP) is a framework
applicable to many real world problems. In this work, we propose an approach to
solve POMDPs with multimodal belief by relying on a policy that solves the
fully observable version. By defininig a new, mixture value function based on
the value function from the fully observable variant, we can use the
corresponding greedy policy to solve the POMDP itself. We develop the
mathematical framework necessary for discussion, and introduce a benchmark
built on the task of Reconnaissance Blind TicTacToe. On this benchmark, we show
that our policy outperforms policies ignoring the existence of multiple modes.
