  Large language models (LLMs) have attracted considerable attention as they
are capable of showcasing impressive capabilities generating comparable
high-quality responses to human inputs. LLMs, can not only compose textual
scripts such as emails and essays but also executable programming code.
Contrary, the automated reasoning capability of these LLMs in performing
statistically-driven descriptive analysis, particularly on user-specific data
and as personal assistants to users with limited background knowledge in an
application domain who would like to carry out basic, as well as advanced
statistical and domain-specific analysis is not yet fully explored. More
importantly, the performance of these LLMs has not been compared and discussed
in detail when domain-specific data analysis tasks are needed. This study,
consequently, explores whether LLMs can be used as generative AI-based personal
assistants to users with minimal background knowledge in an application domain
infer key data insights. To demonstrate the performance of the LLMs, the study
reports a case study through which descriptive statistical analysis, as well as
Natural Language Processing (NLP) based investigations, are performed on a
number of phishing emails with the objective of comparing the accuracy of the
results generated by LLMs to the ones produced by analysts. The experimental
results show that LangChain and the Generative Pre-trained Transformer (GPT-4)
excel in numerical reasoning tasks i.e., temporal statistical analysis, achieve
competitive correlation with human judgments on feature engineering tasks while
struggle to some extent on domain specific knowledge reasoning, where
domain-specific knowledge is required.
