  In this paper, we proposed a conceptual model to predict the chat experience
in a natural language generation dialog system. We evaluated the model with 120
participants with Partial Least Squares Structural Equation Modeling (PLS-SEM)
and obtained an R-square (R2) with 0.541. The model considers various factors,
including the prompts used for generation; coherence, sentiment, and similarity
in the conversation; and users' perceived dialog agents' favorability. We then
further explore the effectiveness of the subset of our proposed model. The
results showed that users' favorability and coherence, sentiment, and
similarity in the dialogue are positive predictors of users' chat experience.
Moreover, we found users may prefer dialog agents with characteristics of
Extroversion, Openness, Conscientiousness, Agreeableness, and Non-Neuroticism.
Through our research, an adaptive dialog system might use collected data to
infer factors in our model, predict the chat experience for users through these
factors, and optimize it by adjusting prompts.
