  Chatbots' role in fostering self-reflection is now widely recognized,
especially in inducing users' behavior change. While the benefits of 24/7
availability, scalability, and consistent responses have been demonstrated in
contexts such as healthcare and tutoring to help one form a new habit, their
utilization in coaching necessitating deeper introspective dialogue to induce
leadership growth remains unexplored. This paper explores the potential of such
a chatbot powered by recent Large Language Models (LLMs) in collaboration with
professional coaches in the field of executive coaching. Through a design
workshop with them and two weeks of user study involving ten coach-client
pairs, we explored the feasibility and nuances of integrating chatbots to
complement human coaches. Our findings highlight the benefits of chatbots'
ubiquity and reasoning capabilities enabled by LLMs while identifying their
limitations and design necessities for effective collaboration between human
coaches and chatbots. By doing so, this work contributes to the foundation for
augmenting one's self-reflective process with prevalent conversational agents
through the human-in-the-loop approach.
