  The increasing progress in artificial intelligence and respective machine
learning technology has fostered the proliferation of chatbots to the point
where today they are being embedded into various human-technology interaction
tasks. In enterprise contexts, the use of chatbots seeks to reduce labor costs
and consequently increase productivity. For simple, repetitive customer service
tasks such already proves beneficial, yet more complex collaborative knowledge
work seems to require a better understanding of how the technology may best be
integrated. Particularly, the additional mental burden which accompanies the
use of these natural language based artificial assistants, often remains
overlooked. To this end, cognitive load theory implies that unnecessary use of
technology can induce additional extrinsic load and thus may have a contrary
effect on users' productivity. The research presented in this paper thus
reports on a study assessing cognitive load and productivity implications of
human chatbot interaction in a realistic enterprise setting. A/B testing
software-only vs. software + chatbot interaction, and the NASA TLX were used to
evaluate and compare the cognitive load of two user groups. Results show that
chatbot users experienced less cognitive load and were more productive than
software-only users. Furthermore, they show lower frustration levels and better
overall performance (i.e, task quality) despite their slightly longer average
task completion time.
