  Conversational Assistants (CA) are increasingly supporting human workers in
knowledge management. Traditionally, CAs respond in specific ways to predefined
user intents and conversation patterns. However, this rigidness does not handle
the diversity of natural language well. Recent advances in natural language
processing, namely Large Language Models (LLMs), enable CAs to converse in a
more flexible, human-like manner, extracting relevant information from texts
and capturing information from expert humans but introducing new challenges
such as ``hallucinations''. To assess the potential of using LLMs for knowledge
management tasks, we conducted a user study comparing an LLM-based CA to an
intent-based system regarding interaction efficiency, user experience,
workload, and usability. This revealed that LLM-based CAs exhibited better user
experience, task completion rate, usability, and perceived performance than
intent-based systems, suggesting that switching NLP techniques can be
beneficial in the context of knowledge management.
