  Recent investigations show that large language models (LLMs), specifically
GPT-4, not only have remarkable capabilities in common Natural Language
Processing (NLP) tasks but also exhibit human-level performance on various
professional and academic benchmarks. However, whether GPT-4 can be directly
used in practical applications and replace traditional artificial intelligence
(AI) tools in specialized domains requires further experimental validation. In
this paper, we explore the potential of LLMs such as GPT-4 to outperform
traditional AI tools in dementia diagnosis. Comprehensive comparisons between
GPT-4 and traditional AI tools are conducted to examine their diagnostic
accuracy in a clinical setting. Experimental results on two real clinical
datasets show that, although LLMs like GPT-4 demonstrate potential for future
advancements in dementia diagnosis, they currently do not surpass the
performance of traditional AI tools. The interpretability and faithfulness of
GPT-4 are also evaluated by comparison with real doctors. We discuss the
limitations of GPT-4 in its current state and propose future research
directions to enhance GPT-4 in dementia diagnosis.
