  While LLMs are proficient at processing text in human conversations, they
often encounter difficulties with the nuances of verbal instructions and, thus,
remain prone to hallucinate trust in human command. In this work, we present
TrustNavGPT, an LLM based audio guided navigation agent that uses affective
cues in spoken communication elements such as tone and inflection that convey
meaning beyond words, allowing it to assess the trustworthiness of human
commands and make effective, safe decisions. Our approach provides a
lightweight yet effective approach that extends existing LLMs to model audio
vocal features embedded in the voice command and model uncertainty for safe
robotic navigation.
