  Situational awareness is a critical foundation for the protection of human
life/properties and is challenging to maintain for people with disabilities
(i.e., visual impairments and limited mobility). In this paper, we present a
dialog enabled end-to-end assistive solution called "SafeAccess" to build a
safer smart home by providing situational awareness. The key functions of
SafeAccess are: - 1) monitoring homes and identifying incoming persons; 2)
helping users in assessing incoming threats (e.g., burglary, robbery, gun
violence); and, 3) allowing users to grant safe access to homes for
friends/families. In this work, we focus on building a robust model for
detecting and recognizing person, generating image descriptions, and designing
a prototype for the smart door. To interact with the system, we implemented a
dialog enabled smartphone app, especially for creating a personalized profile
from face images or videos of friends/families. A Raspberry pi connected to the
home monitoring cameras captures the video frames and performs change detection
to identify frames with activities. Then, we detect human presence using Faster
r-cnn and extract faces using Multi-task Cascaded Convolutional Networks
(MTCNN). Subsequently, we match the detected faces using FaceNet/support vector
machine (SVM) classifiers. The system notifies users with an MMS containing the
name of incoming persons or as "unknown", scene image, facial description, and
contextual information. The users can grant access or call emergency services
using the SafeAccess app based on the received notification. Our system
identifies persons with an F-score 0.97 and recognizes items to generate image
description with an average F-score 0.97.
