  Momentive offers solutions in market research, customer experience, and
enterprise feedback. The technology is gleaned from the billions of real
responses to questions asked on the platform. However, people may create biased
questions. A double-barreled question (DBQ) is a common type of biased question
that asks two aspects in one question. For example, "Do you agree with the
statement: The food is yummy, and the service is great.". This DBQ confuses
survey respondents because there are two parts in a question. DBQs impact both
the survey respondents and the survey owners. Momentive aims to detect DBQs and
recommend survey creators to make a change towards gathering high quality
unbiased survey data. Previous research work has suggested detecting DBQs by
checking the existence of grammatical conjunction. While this is a simple
rule-based approach, this method is error-prone because conjunctions can also
exist in properly constructed questions. We present an end-to-end machine
learning approach for DBQ classification in this work. We handled this
imbalanced data using active learning, and compared state-of-the-art embedding
algorithms to transform text data into vectors. Furthermore, we proposed a
model interpretation technique propagating the vector-level SHAP values to a
SHAP value for each word in the questions. We concluded that the word2vec
subword embedding with maximum pooling is the optimal word embedding
representation in terms of precision and running time in the offline
experiments using the survey data at Momentive. The A/B test and production
metrics indicate that this model brings a positive change to the business. To
the best of our knowledge, this is the first machine learning framework for DBQ
detection, and it successfully differentiates Momentive from the competitors.
We hope our work sheds light on machine learning approaches for bias question
detection.
