  This study evaluates the effectiveness of various large language models
(LLMs) in performing tasks common among undergraduate computer science
students. Although a number of research studies in the computing education
community have explored the possibility of using LLMs for a variety of tasks,
there is a lack of comprehensive research comparing different LLMs and
evaluating which LLMs are most effective for different tasks. Our research
systematically assesses some of the publicly available LLMs such as Google
Bard, ChatGPT(3.5), GitHub Copilot Chat, and Microsoft Copilot across diverse
tasks commonly encountered by undergraduate computer science students in India.
These tasks include code explanation and documentation, solving class
assignments, technical interview preparation, learning new concepts and
frameworks, and email writing. Evaluation for these tasks was carried out by
pre-final year and final year undergraduate computer science students and
provides insights into the models' strengths and limitations. This study aims
to guide students as well as instructors in selecting suitable LLMs for any
specific task and offers valuable insights on how LLMs can be used
constructively by students and instructors.
