  With increasing physical threats in recent years targeted at critical
infrastructures, it is crucial to establish a reliable threat monitoring system
integrating video surveillance and digital sensors based on cutting-edge
technologies. A physical threat monitoring solution unifying the floorplan,
cameras, and sensors for smart buildings has been set up in our study. Computer
vision and deep learning models are used for video streams analysis. When a
threat is detected by a rule engine based on the real-time analysis results
combining with feedback from related digital sensors, an alert is sent to the
Video Management System so that human operators can take further action. A
physical threat monitoring system typically needs to address complex and even
destructive incidents, such as fire, which is unrealistic to simulate in real
life. Restrictions imposed during the Covid-19 pandemic and privacy concerns
have added to the challenges. Our study utilises the Unreal Engine to simulate
some typical suspicious and intrusion scenes with photorealistic qualities in
the context of a virtual building. Add-on programs are implemented to transfer
the video stream from virtual PTZ cameras to the Milestone Video Management
System and enable users to control those cameras from the graphic client
application. Virtual sensors such as fire alarms, temperature sensors and door
access controls are implemented similarly, fulfilling the same programmatic VMS
interface as real-life sensors. Thanks to this simulation system's
extensibility and repeatability, we have consolidated this unified physical
threat monitoring system and verified its effectiveness and user-friendliness.
Both the simulated Unreal scenes and the software add-ons developed during this
study are highly modulated and thereby are ready for reuse in future projects
in this area.
