  Integrating Generative AI (GenAI) into educational contexts presents a
transformative potential for enhancing learning experiences. This paper
introduces CourseGPT, a generative AI tool designed to support instructors and
enhance the educational experiences of undergraduate students. Built on
open-source Large Language Models (LLMs) from Mistral AI, CourseGPT offers
continuous instructor support and regular updates to course materials,
enriching the learning environment. By utilizing course-specific content, such
as slide decks and supplementary readings and references, CourseGPT provides
precise, dynamically generated responses to student inquiries. Unlike generic
AI models, CourseGPT allows instructors to manage and control the responses,
thus extending the course scope without overwhelming details. The paper
demonstrates the application of CourseGPT using the CPR E 431 - Basics of
Information System Security course as a pilot. This course, with its large
enrollments and diverse curriculum, serves as an ideal testbed for CourseGPT.
The tool aims to enhance the learning experience, accelerate feedback
processes, and streamline administrative tasks. The study evaluates CourseGPT's
impact on student outcomes, focusing on correctness scores, context recall, and
faithfulness of responses. Results indicate that the Mixtral-8x7b model, with a
higher parameter count, outperforms smaller models, achieving an 88.0%
correctness score and a 66.6% faithfulness score. Additionally, feedback from
former students and teaching assistants on CourseGPT's accuracy, helpfulness,
and overall performance was collected. The outcomes revealed that a significant
majority found CourseGPT to be highly accurate and beneficial in addressing
their queries, with many praising its ability to provide timely and relevant
information.
