  Understanding and addressing unfairness in LLMs are crucial for responsible
AI deployment. However, there is a limited number of quantitative analyses and
in-depth studies regarding fairness evaluations in LLMs, especially when
applying LLMs to high-stakes fields. This work aims to fill this gap by
providing a systematic evaluation of the effectiveness and fairness of LLMs
using ChatGPT as a study case. We focus on assessing ChatGPT's performance in
high-takes fields including education, criminology, finance and healthcare. To
conduct a thorough evaluation, we consider both group fairness and individual
fairness metrics. We also observe the disparities in ChatGPT's outputs under a
set of biased or unbiased prompts. This work contributes to a deeper
understanding of LLMs' fairness performance, facilitates bias mitigation and
fosters the development of responsible AI systems.
