  Stack Overflow, the world's largest software Q&A (SQA) website, is facing a
significant traffic drop due to the emergence of generative AI techniques.
ChatGPT is banned by Stack Overflow after only 6 days from its release. The
main reason provided by the official Stack Overflow is that the answers
generated by ChatGPT are of low quality. To verify this, we conduct a
comparative evaluation of human-written and ChatGPT-generated answers. Our
methodology employs both automatic comparison and a manual study. Our results
suggest that human-written and ChatGPT-generated answers are semantically
similar, however, human-written answers outperform ChatGPT-generated ones
consistently across multiple aspects, specifically by 10% on the overall score.
We release the data, analysis scripts, and detailed results at
https://anonymous.4open.science/r/GAI4SQA-FD5C.
