  Commit messages are explanations of changes made to a codebase that are
stored in version control systems. They help developers understand the codebase
as it evolves. However, writing commit messages can be tedious and inconsistent
among developers. To address this issue, researchers have tried using different
methods to automatically generate commit messages, including rule-based,
retrieval-based, and learning-based approaches. Advances in large language
models offer new possibilities for generating commit messages. In this study,
we evaluate the performance of OpenAI's ChatGPT for generating commit messages
based on code changes. We compare the results obtained with ChatGPT to previous
automatic commit message generation methods that have been trained specifically
on commit data. Our goal is to assess the extent to which large pre-trained
language models can generate commit messages that are both quantitatively and
qualitatively acceptable. We found that ChatGPT was able to outperform previous
Automatic Commit Message Generation (ACMG) methods by orders of magnitude, and
that, generally, the messages it generates are both accurate and of
high-quality. We also provide insights, and a categorization, for the cases
where it fails.
