  Audio commands are a preferred communication medium to keep inspectors in the
loop of civil infrastructure inspection performed by a semi-autonomous drone.
To understand job-specific commands from a group of heterogeneous and dynamic
inspectors, a model must be developed cost-effectively for the group and easily
adapted when the group changes. This paper is motivated to build a
multi-tasking deep learning model that possesses a Share-Split-Collaborate
architecture. This architecture allows the two classification tasks to share
the feature extractor and then split subject-specific and keyword-specific
features intertwined in the extracted features through feature projection and
collaborative training. A base model for a group of five authorized subjects is
trained and tested on the inspection keyword dataset collected by this study.
The model achieved a 95.3% or higher mean accuracy in classifying the keywords
of any authorized inspectors. Its mean accuracy in speaker classification is
99.2%. Due to the richer keyword representations that the model learns from the
pooled training data, adapting the base model to a new inspector requires only
a little training data from that inspector, like five utterances per keyword.
Using the speaker classification scores for inspector verification can achieve
a success rate of at least 93.9% in verifying authorized inspectors and 76.1%
in detecting unauthorized ones. Further, the paper demonstrates the
applicability of the proposed model to larger-size groups on a public dataset.
This paper provides a solution to addressing challenges facing AI-assisted
human-robot interaction, including worker heterogeneity, worker dynamics, and
job heterogeneity.
