ICLS 2010 Pre-conference Workshop
Tuesday, June 29, 9:00AM – 12:30PM
Intelligent tutoring systems guide learners as they practice a complex cognitive skill. They have been shown to enhance learning in a range of domains, and are increasingly being used as platforms for learning science experiments. This workshop provides a hands-on tutorial introduction to building tutors using the freely available Cognitive Tutor Authoring Tools (CTAT). Using CTAT, authors can create a new type of tutors, example-tracing tutors, without programming. No background in computer science is required.
Associate Professor of Human-Computer Interaction
Human-Computer Interaction Institute
Carnegie Mellon University
5000 Forbes Ave.
Pittsburgh, PA 15213
The workshop will help make intelligent tutoring systems technology available to educational researchers who may find that the tools make tutor development and web delivery sufficiently easy for them to consider using the technology.
Goals: The workshop will provide:
Novice users without programming experience have found that they can get started with CTAT, and learn to create a tutor for a simple task, in the course of half a day. This workshop is meant to give learning sciences researchers a thorough introduction to these tools, so they can evaluate whether it might be a useful tool for them.
It is assumed that all participants will bring laptops with pre-installed CTAT software. This software can be downloaded free of charge (provided it will be used for non-commercial purposes) from http://ctat.pact.cs.cmu.edu.
All hands-on activities will involve carefully prepared examples of partially-finished tutors that the participants will complete. In the hands-on activities, workshop participants will extend the tutors shown in the demonstration activities.
Vincent Aleven is an Associate Professor in Carnegie Mellon’s Human-Computer Interaction Institute, and has over 20 years of experience in research and development of educational software based on cognitive science theory. He has published in journals such as Cognitive Science, Review of Educational Research, Educational Psychology Review, International Journal of Artificial Intelligence in Education, Instructional Science, and Artificial Intelligence. He and colleagues have created the CTAT authoring tools that enable non-programmers to create tutors much more cost-effectively than programmers used to create them. His research interests also include the scaling of intelligent tutoring systems for real-world use and the role of metacognition in learning with intelligent tutoring systems. Aleven is a member of the Executive Committee of the Pittsburgh Science of Learning Center (PSLC). Aleven has co-designed and co-operated the PSLC summer school for the past 9 years. He is the Program Co-Chair of the 2010 International Conference on Intelligent Tutoring Systems. He has been or is PI on four major research grants and co-PI on 6 others.
Jonathan Sewall is Project Director in the Human-Computer Interaction Institute at Carnegie Mellon University. He has 25 years of experience in government, industry and academia with software design and development. He has been the technical lead on the Cognitive Tutor Authoring Tools project for the last 4 years. His past work experience includes expert systems, massively parallel systems, web applications, databases and networks.
Intelligent tutoring systems (ITS) are an established educational technology (VanLehn, 2006; Woolf, 2008) and are increasingly being used as platform for learning sciences research. Systems of this type provide step-by-step guidance as students practice solving complex cognitive skill. They typically support learners by means of (a) a user interface carefully designed to make thinking visible, (b) feedback on individual solution steps and not just the final solution to a problem, and (c) context-sensitive next-step hints, which communicate which problem-solving principles apply and how. Many ITS also (d) select problems on an individual basis, based on a detailed (automated) assessment of each individual student’s problem-solving skill. A number of research studies have shown that ITS help students achieve better learning results compared to comparison curricula (e.g., Koedinger, Anderson, Hadley & Mark, 1997; Koedinger & Aleven, 2007).
ITS are convenient platforms for learning sciences research. For example, to test an instructional principle (e.g., the worked examples principle) a researcher might compare student learning with two different versions of the same tutor, one that implements the principle and one that does not (McLaren, Lim & Koedinger, 2008; Rau, Aleven & Rummel, 2009). The tutoring software can deliver instructional treatments in a highly consistent manner. Logs of student-tutor interactions provide a wealth of longitudinal data about student learning, and often shed light on how the different tutor versions affect learning.
The Cognitive Tutor Authoring Tools (CTAT) facilitate the development of web-based intelligent tutors without programming and may therefore be an attractive tool for learning sciences researchers. They support development of two types of tutors: Cognitive Tutors and a newer type of tutors called example-tracing tutors. Both are grounded in the ACT-R theory of cognition (Anderson & Lebière, 1998).
Figure 1. The CTAT tools during a session authoring example-tracing tutors, with (left to right), the Adobe Flash drag-and-drop interface development tool, the student interface created with that tool, and an editable behavior graph, which records correct solution paths demonstrated by the author and is used to provide tutoring.
Example-tracing tutors can be built through programming-by-demonstration (Aleven, McLaren, Sewall & Koedinger, 2009). An author (see Figure 1) creates a student interface through drag-and-drop techniques and then demonstrates solution paths for problems for which the tutor is to provide tutoring. The tools record the demonstrated solution paths in an editable behavior graph, which the author can generalize in various ways and can annotate with hints and skill labels. When running in tutoring mode, CTAT uses the generalized behavior graph to evaluate student behavior and to provide next-step hints. The generalization features make it possible for the tutor to recognize a wider range of behavior as correct than just the recorded solution paths. Data from over 26 research studies that have used CTAT indicate that the tools lower the cost of intelligent tutor development by a factor of 4 through 8.
CTAT has a number of features that facilitate its use in building tutors for learning sciences research. Chief among these is that all CTAT-built tutors automatically generate rich log data of student-tutor interactions. These log data can be analyzed using the DataShop (Koedinger, Cunningham, Skogsholm & Leber, 2008), a suite of web-based data analysis tools geared toward analysis of “learning curves” distilled from tutor data (http://pslcdatashop.org). Secondly, as a practical matter, speed and ease of tutor development can be significant gains to research efforts, especially those with limited time or budgets. Since example-tracing tutors can be built without programming, researchers need not obtain expensive programming expertise to create their experimental materials. And since the tools are easy to learn and use, researchers can easily and quickly build new versions of a tutor, (e.g. after pilot testing) to correct for unexpected problems or test new conditions.
CTAT has been used to build tutors for a variety of domains, including middle-school mathematics (Aleven, McLaren & Sewall, 2009), elementary stoichiometry, foreign language grammar instruction such as English articles, Chinese character recognition and writing, and intercultural competence (Ogan, Aleven & Jones, in press). CTAT has also been integrated with existing simulation environments such as a simulator for thermodynamic cycles (http://www.qrg.northwestern.edu/projects/NSF/cyclepad/cyclepad.htm) and one for mixing and heating solutions on a chemistry bench (http://www.chemcollective.org/vlab/vlab.php). The CTAT tutoring adds goal-oriented hints and feedback to these exploratory environments.
Aleven, V., McLaren, B. M., & Sewall, J. (2009). Scaling up programming by demonstration for intelligent tutoring systems development: An open-access website for middle-school mathematics learning. IEEE Transactions on Learning Technologies, 2(2), 64-78.
Aleven, V., McLaren, B. M., Sewall, J., & Koedinger, K. R. (2009). A new paradigm for intelligent tutoring systems: Example-tracing tutors. International Journal of Artificial Intelligence in Education, 19(2), 105-154.
Anderson, J. R., & Lebiere, C. (1998). The atomic components of thought. Mahwah, NJ: Lawrence Erlbaum Associates.
Koedinger, K. R., & Aleven V. (2007). Exploring the assistance dilemma in experiments with Cognitive Tutors. Educational Psychology Review, 19(3), 239-264.
Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8(1), 30-43.
Koedinger, K., Cunningham, K., Skogsholm A., & Leber, B. (2008). An open repository and analysis tools for fine-grained, longitudinal learner data. In R.S.J.d. Baker, T. Barnes, & J. E. Beck, (Eds.), , Proceedings of the 1st International Conference on Educational Data Mining (pp. 157-166). Montreal, Canada.
McLaren, B. M., Lim, S. J., & Koedinger, K. R. (2008). When and how often should worked examples be given to students? New results and a summary of the current state of research. In B. C. Love, K. McRae, & V. M. Sloutsky (Eds.), Proceedings of the 30th annual meeting of the cognitive science society. Austin, TX: Cognitive Science Society.
Ogan, A., Aleven, V., & Jones, C. (in press). Advancing development of intercultural competence through supporting predictions in narrative video. International Journal of Artificial Intelligence in Education.
Rau, M., Aleven, V., & Rummel, N. (2009). Intelligent tutoring systems with multiple representations and self-explanation prompts support learning of fractions. In V. Dimitrova, R. Mizoguchi, B. du Boulay, & A. Graesser (Eds.), Proceedings of the 14th International Conference on Artificial Intelligence in Education, AIED 2009 (pp. 441-448). Amsterdam: IOS Press.
VanLehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16(3), 227-265.
Woolf, B. P. (2008). Building intelligent interactive tutors: Student-Centered strategies for revolutionizing e-learning. Morgan Kaufmann.
The development of the CTAT Tools was funded by grants from the Office of Naval Research and the Grable Foundation and an NSF grant to the Pittsburgh Science of Learning Center, award number SBE-0836012.