  This work investigates large language models (LLMs) as teachable agents for
learning by teaching (LBT). LBT with teachable agents helps learners identify
knowledge gaps and discover new knowledge. However, teachable agents require
expensive programming of subject-specific knowledge. While LLMs as teachable
agents can reduce the cost, LLMs' expansive knowledge as tutees discourages
learners from teaching. We propose a prompting pipeline that restrains LLMs'
knowledge and makes them initiate "why" and "how" questions for effective
knowledge-building. We combined these techniques into TeachYou, an LBT
environment for algorithm learning, and AlgoBo, an LLM-based tutee chatbot that
can simulate misconceptions and unawareness prescribed in its knowledge state.
Our technical evaluation confirmed that our prompting pipeline can effectively
configure AlgoBo's problem-solving performance. Through a between-subject study
with 40 algorithm novices, we also observed that AlgoBo's questions led to
knowledge-dense conversations (effect size=0.71). Lastly, we discuss design
implications, cost-efficiency, and personalization of LLM-based teachable
agents.
