  In recent years, large language models (LLMs), such as ChatGPT, have been
pivotal in advancing various artificial intelligence applications, including
natural language processing and software engineering. A promising yet
underexplored area is utilizing LLMs in software testing, particularly in
black-box testing. This paper explores the test cases devised by ChatGPT in
comparison to those created by human participants. In this study, ChatGPT
(GPT-4) and four participants each created black-box test cases for three
applications based on specifications written by the authors. The goal was to
evaluate the real-world applicability of the proposed test cases, identify
potential shortcomings, and comprehend how ChatGPT could enhance human testing
strategies. ChatGPT can generate test cases that generally match or slightly
surpass those created by human participants in terms of test viewpoint
coverage. Additionally, our experiments demonstrated that when ChatGPT
cooperates with humans, it can cover considerably more test viewpoints than
each can achieve alone, suggesting that collaboration between humans and
ChatGPT may be more effective than human pairs working together. Nevertheless,
we noticed that the test cases generated by ChatGPT have certain issues that
require addressing before use.
