  Effective decision-making in autonomous driving relies on accurate inference
of other traffic agents' future behaviors. To achieve this, we propose an
online belief-update-based behavior prediction model and an efficient planner
for Partially Observable Markov Decision Processes (POMDPs). We develop a
Transformer-based prediction model, enhanced with a recurrent neural memory
model, to dynamically update latent belief state and infer the intentions of
other agents. The model can also integrate the ego vehicle's intentions to
reflect closed-loop interactions among agents, and it learns from both offline
data and online interactions. For planning, we employ a Monte-Carlo Tree Search
(MCTS) planner with macro actions, which reduces computational complexity by
searching over temporally extended action steps. Inside the MCTS planner, we
use predicted long-term multi-modal trajectories to approximate future updates,
which eliminates iterative belief updating and improves the running efficiency.
Our approach also incorporates deep Q-learning (DQN) as a search prior, which
significantly improves the performance of the MCTS planner. Experimental
results from simulated environments validate the effectiveness of our proposed
method. The online belief update model can significantly enhance the accuracy
and temporal consistency of predictions, leading to improved decision-making
performance. Employing DQN as a search prior in the MCTS planner considerably
boosts its performance and outperforms an imitation learning-based prior.
Additionally, we show that the MCTS planning with macro actions substantially
outperforms the vanilla method in terms of performance and efficiency.
