  Previous studies in the perimeter defense game have largely focused on the
fully observable setting where the true player states are known to all players.
However, this is unrealistic for practical implementation since defenders may
have to perceive the intruders and estimate their states. In this work, we
study the perimeter defense game in a photo-realistic simulator and the real
world, requiring defenders to estimate intruder states from vision. We train a
deep machine learning-based system for intruder pose detection with domain
randomization that aggregates multiple views to reduce state estimation errors
and adapt the defensive strategy to account for this. We newly introduce
performance metrics to evaluate the vision-based perimeter defense. Through
extensive experiments, we show that our approach improves state estimation, and
eventually, perimeter defense performance in both 1-defender-vs-1-intruder
games, and 2-defenders-vs-1-intruder games.
