  Existing simulations designed for cultural and interpersonal skill training
rely on pre-defined responses with a menu option selection interface. Using a
multiple-choice interface and restricting trainees' responses may limit the
trainees' ability to apply the lessons in real life situations. This systems
also uses a simplistic evaluation model, where trainees' selected options are
marked as either correct or incorrect. This model may not capture sufficient
information that could drive an adaptive feedback mechanism to improve
trainees' cultural awareness. This paper describes the design of a
dialogue-based simulation for cultural awareness training. The simulation,
built around a disaster management scenario involving a joint coalition between
the US and the Chinese armies. Trainees were able to engage in realistic
dialogue with the Chinese agent. Their responses, at different points, get
evaluated by different multi-label classification models. Based on training on
our dataset, the models score the trainees' responses for cultural awareness in
the Chinese culture. Trainees also get feedback that informs the cultural
appropriateness of their responses. The result of this work showed the
following; i) A feature-based evaluation model improves the design, modeling
and computation of dialogue-based training simulation systems; ii) Output from
current automatic speech recognition (ASR) systems gave comparable end results
compared with the output from manual transcription; iii) A multi-label
classification model trained as a cultural expert gave results which were
comparable with scores assigned by human annotators.
