  In this study, we aim to find a method to auto-tag sentences specific to a
domain. Our training data comprises short conversational sentences extracted
from chat conversations between company's customer representatives and web site
visitors. We manually tagged approximately 14 thousand visitor inputs into ten
basic categories, which will later be used in a transformer-based language
model with attention mechanisms for the ultimate goal of developing a chatbot
application that can produce meaningful dialogue. We considered three different
state-of-the-art models and reported their auto-tagging capabilities. We
achieved the best performance with the bidirectional encoder representation
from transformers (BERT) model. Implementation of the models used in these
experiments can be cloned from our GitHub repository and tested for similar
auto-tagging problems without much effort.
