  Automatic summarization of legal case judgements has traditionally been
attempted by using extractive summarization methods. However, in recent years,
abstractive summarization models are gaining popularity since they can generate
more natural and coherent summaries. Legal domain-specific pre-trained
abstractive summarization models are now available. Moreover, general-domain
pre-trained Large Language Models (LLMs), such as ChatGPT, are known to
generate high-quality text and have the capacity for text summarization. Hence
it is natural to ask if these models are ready for off-the-shelf application to
automatically generate abstractive summaries for case judgements. To explore
this question, we apply several state-of-the-art domain-specific abstractive
summarization models and general-domain LLMs on Indian court case judgements,
and check the quality of the generated summaries. In addition to standard
metrics for summary quality, we check for inconsistencies and hallucinations in
the summaries. We see that abstractive summarization models generally achieve
slightly higher scores than extractive models in terms of standard summary
evaluation metrics such as ROUGE and BLEU. However, we often find inconsistent
or hallucinated information in the generated abstractive summaries. Overall,
our investigation indicates that the pre-trained abstractive summarization
models and LLMs are not yet ready for fully automatic deployment for case
judgement summarization; rather a human-in-the-loop approach including manual
checks for inconsistencies is more suitable at present.
