  Large Language Models (LLMs) have gathered significant attention due to their
impressive performance on a variety of tasks. ChatGPT, developed by OpenAI, is
a recent addition to the family of language models and is being called a
disruptive technology by a few, owing to its human-like text-generation
capabilities. Although, many anecdotal examples across the internet have
evaluated ChatGPT's strength and weakness, only a few systematic research
studies exist. To contribute to the body of literature of systematic research
on ChatGPT, we evaluate the performance of ChatGPT on Abstractive Summarization
by the means of automated metrics and blinded human reviewers. We also build
automatic text classifiers to detect ChatGPT generated summaries. We found that
while text classification algorithms can distinguish between real and generated
summaries, humans are unable to distinguish between real summaries and those
produced by ChatGPT.
