Conference Room, 406)
This course counts toward the Intelligent
Systems (formerly AI) track credit for PhD students in the IS track.
It also counts toward MS credit for MS students enrolled in the
Intelligent Robotics MS program (MSCS-IR) .
Undergraduate AI (e.g., CSCI 460) or graduate
AI (CS 561) or Introduction to Robotics (CSCI 445)
Sensing and Planning
in Robotics (CSCI 547)
or permission of instructor.
Survey of robot control and learning methods from technical papers.
Control architectures, adaptation, learning, cooperation, distributed
v. centralized approaches, cooperative and competitive systems.
Prerequisite: CSCI 460 or 445 or 561 or 547.
This course is a seminar-style hands-on survey of approaches to
control and learning in single and multi-robot systems. We will read
original seminal papers that track the development of the field and
overview the different state-of-the-art approaches to mobile robot
control, including reactive, hybrid, and behavior-based based systems.
The discussion will focus on the issues of resolving the fundamental
conflict between thinking and acting, i.e., high-level deliberation
and real-time control. Different approaches and robot control
architectures for addressing this issue will be covered and discussed.
In the second part of the course we will discuss scaling up robot
control to multi-robot systems and swarms of robots. The control
architectures discussed in the first part of the semester will be
revisited in the context of scaling up to distributed systems.
Finally, we will address adaptation and learning in single and
multi-robot systems, and deal with the many challenges those problems
present. Several other relevant topics will be covered at least
briefly, including biological inspirations for robot control and
philosophical foundations. All topics will be illustrated with
implemented systems and demonstrated with videos.
Each week, all students will read all of the assigned readings. Each
of the assigned readings will be presented by a student, and discussed
and critiqued by all others. The student presenting a paper in a given
class should prepare a clear 30-minute presentation. This
presentation should assume that the audience has read the paper, and
not spend more than about 5 minutes summarizing it. Most of the
presentation should be spent on discussing the paper, its strengths,
weaknesses, any points needing clarification, and addressing any
questions the other students may have.
For information on how to present a paper, look here
You are expected to prepare a PowerPoint/Foils presentation to show
from your laptop. An LCD projector will be available in the meeting
room for the paper presentations; it is your responsibility to provide
the presentation on a laptop that is compatible with the projector.
For any logistical questions, please contact the course TA ahead of
For each paper being covered on a given day, each student (whether
presenting or not), will bring to class a brief summary report,
consisting of the following parts:
- a paragraph briefly summarizing the contributions of the paper (copies of
the abstract and/or intro and/or conclusions will not do);
- a paragraph critiquing the paper. The critique should take the form of
addressing the strengths and weaknesses of the paper, comparing it briefly to
other work, if relevant, etc.
All summary reports should include: the student's
name, and the title and authors of the paper being summarized and
critiqued. No hand-written reports can be accepted.
In summary, for each class, you should bring:
- the papers being discussed,
- a summary report for each of the papers,
- a presentation, if you are presenting one of the papers,
- great enthusiasm for discussing all of the papers. :)
RP) is the background textbook for this course. In press by MIT
Press, it will be available to you at the start of the semester. The
material in this textbook is assumed background for the course. You
can read the textbook at the start of the semester or throughout (as
indicated by the syllabus); if you have prior robotics background much
of this material should be familiar to you already. In any case, you
are expected to have read the textbook, which is designed as
background and is fast and easy to read, so you will be familiar with
the basic material that is the foundation for the course.
The course readings consist of published papers from the field, listed
below. Papers marked with a black dot will be presented; you will be
expected to read (or view, if videos) the background material (marked
with ) as well, as it supports the papers. For additional optional
papers and resources, please refer to the list found
here. Three optional background textbooks
RP: chapters 1-10
RP: chapters 11 & 12
RP: chapter 13
- "Intelligence Without Reason", Rodney A. Brooks, Proceedings of 12th
International Joint Conference on Artificial Intelligence (IJCAI-91),
Sydney, Australia, pages 569-595, August 1991.
Presenter: Prof. Mataric´
- "Modeling Adaptive Autonomous Agents", Pattie Maes, Artificial
Life, 1(1-2), MIT Press, pages 135-162, 1994.
Presenter: Prof. Mataric´
- "What are Plans For", Philip Agre and David Chapman, Robotics and Autonomous Systems, vol. 6, Elsevier Science, pages 17-34, 1990.
- "A Robust Layered Control System for a Mobile Robot", Rodney A. Brooks, IEEE Transactions on Robotics and Automation, 2(1), pages
14-23, April 1986.
- "A Robot that Walks; Emergent Behavior from a Carefully Evolved Network", Rodney A. Brooks, Neural Computation, 1(2), MIT Press,
pages 253-262, 1989.
- "Universal Plans for Reactive Robots in Unpredictable Environments",
Marcel Schoppers, Proceedings of the 10th International Joint Conference on Artificial Intelligence (IJCAI-87),
Milan, Italy, pages 1039-1046, August 1987.
Here is an inflammatory debate about the paper above:
RP: chapters 14 and 15
RP: chapter 16
Here is a brief background article on control architectures and representation in behavior-based robotics:
"SSS: A Hybrid Architecture Applied to Robot Navigation", Jon
Connell, Proceedings, IEEE International Conference on Robotics and
Automation (ICRA-92), Nice, France, pages 2719-2724, May 12-14,
- "On Three-Layer Architectures", Eran Gat, Artificial Intelligence
and Mobile Robotics, in D. Kortenkamp, R. P. Bonnasso and R. Murphy
(eds.), AAAI Press, pages 195-210, 1998.
- Tutorial on Player/Stage/Gazebo.
"Estimating Uncertain Spatial Relationships in Robotics", Peter Cheeseman, Randall C. Smith, Matthew Self,
n Autonomous Robot Vehicles, I. J. Cox and G. T. Wilfong, editors, pp. 167-193, Springer-Verlag, 1990.
- "The Distributed Architecture for Mobile Navigation", Julio Rosenblatt, Journal of Experimental and Theoretical Artificial
Intelligence, 9(2-3), pages 339-360, April 1997.
- "A Survey of Socially Interactive Robots", T. Fong, Illah Nourbakhsh
and Kerstin Dautenhahn, Robotics and Autonomous Systems, 42(3-4),
pages 143-166, 2003.
- "MINERVA: A second generation mobile tour-guide robot", Sebatian
Thrun, Maren Bennewitz, Wolfram Burgard, Armin B. Cremers, Frank Dellaert, Dieter Fox, Dirk Haehnel, Charles Rosenberg, Nicholas Roy, Jamieson Schulte, and Dirk Schulz, Proceedings, IEEE International
Conference on Robotics and Automation (ICRA), 10-15 May 1999.
- "Roball, the rolling robot", Francois Michaud and Serge Caron, Autonomous
Robots, 12(2), pages 211-222, 2002.
- "Towards robotic assistants in nursing homes: challenges and results",
Joelle Pineau, Micheal Montemerlo, Martha Pollack, Nicholas Roy and Sebastian Thrun. Robotics and Autonomous Systems, Volume 42, Issues 3-4, 31 March 2003, pages 271-281.
- The following three short papers (four pages a piece) are grouped together
- "Defining Socially Assistive Robotics",
David J. Feil-Seifer and Maja J. Mataric',
Poster paper in International Conference on Rehabilitation Robotics, Chicago, Illinois, Jun 2005.
- "A Hands-Off Physical Therapy Assistance Robot for Cardiac Patients",
Kyong Il Kang, Sanford Freedman, Maja J. Mataric', Mark J. Cunningham, and Becky Lopez.
Poster paper in International Conference on Rehabilitation Robotics, Chicago, Illinois, Jun 2005.
- "Hands-off Assistive Robotics for Post-Stroke Arm Rehabilitation",
Jon Eriksson, Maja J. Mataric' and Carolee J. Winstein. In International Conference on Rehabilitation Robotics, Chicago, Illinois, Jun 2005.
"A motivation system for regulating human-robot interaction", Cynthia Breazeal, Proceedings of the fifteenth National Conference on Artificial Intelligence (AAAI 98). Madison, WI, 1998, pages 54-61.
RP: chapter TBA
- "Wheelesley, A Robotic Wheelchair System: Indoor Navigation and User Interface.",
Holly A. Yanco, In Lecture Notes in Artificial Intelligence: Assistive
Technology and Artificial Intelligence,, edited by V.O. Mittal, H.A. Yanco,
J. Aronis, and R. Simspon. Springer-Verlag, 1998, pp. 256-268.
- "Mixed-Initiative Control of Multiple Heterogeneous Robots for Urban Search and Rescue"
Robin R. Murphy, Jennifer Casper, Mark Micire, Jeff Hyams. Center for Robot Assisted Search and Rescue Technical Report CRASAR-TR2000-11, 2000.
- "User-Adaptive Control of a Magnetorheological Prosthetic Knee",
Hugh Herr and Ari Wilkenfeld, Industrial Robot: An International Journal 2003; 30: 42-55.
RP: chapter TBA
Background reading on genetic algorithms
Background reading on genetic programming
"Evolution of Subsumption Using Genetic Programming", John Koza, Towards a
Practice of Autonomous Systems: Proceedings of the First European Conference on Artificial Life, F. J. Varela and P. Bourgine (eds.), Paris, France, MIT Press, pages 110-119, 1992.
- "Evolving Electronic Robot Controllers that Exploit Hardware Resources", Arian Thompson, F. Moran, A. Moreno, J. J. Merelo, and P. Chacon (eds.), Advances in Artificial Life: Proceedings of the Third European Conference on Artificial Life, 929, Springer-Verlag, pages 640-656, 1995.
Background reading on coevolution
Background reading on Karl Sims' Virtual Creatures
PC software version of Sims' code is found here
RP: chapter 18
- "Learning to Coordinate Behaviors", Pattie Maes and Rodney A. Brooks, Proceedings, 8th National Conference on Artificial
Intelligence (AAAI-90), AAAI Press/MIT Press, pages 796-802, 1990.
- "Automatic Programming of Behavior-Based Robots Using Reinforcement Learning", Sridhar Mahadevan and Jon Connell, Artificial Intelligence , 55(2-3), 311-365, 1991.
Background reading on ALVINN
Knowledge-based Training of Artificial Neural Networks for Autonomous Robot Driving", Dean Pomerleau, Robot Learning, Kluwer Academic Publishing, pages 19-43, 1993.
RP: chapter TBA
A (partial but extensive) annotated bibliography of work on stigmergy.
"Stigmergy, self-organization, and sorting in collective robotics", Owen Holland and Chris Melhuish, Artificial Life, 5(2), pages 173-202, 1999.
"Cooperation without deliberation: A minimal behavior-based approach
to multi-robot teams", Barry Werger, Artificial Intelligence, 110, pages 293-320, 1999.
A video of the box-pushing robots from the above paper.
"Cooperative transport by ants and robots", C. Ronald Kube and Eric Bonabeau, Robotics and Autonomous Systems 30(1-2), pages 85-101, 2000.
- "Distributed Mobile Robotics by the Method of Dynamic Teams", Jim
Jennings and C. Kirkwood-Watts, Proceedings, International Symposium on Distributed Autonomous Robotic Systems, Karlsruhe,
Germany, May 1998.
"Task decomposition, dynamic role assignment, and low-bandwidth communication for real-time strategic teamwork", Peter Stone and Manuela Veloso, Artificial Intelligence, 110(2), pages 241-273, 1998.
"Dynamic Sensor Planning and Control for Optimally Tracking Targets" by J. Spletzer and C. J. Taylor, International Journal of
Robotics Research,22(1), January 2003, pages 7-20.
RP: chapter TBA
- "Minimalism + Distribution = Supermodularity", by B. Donald, J. Jennings, and D. Rus, Journal of Experimental and Theoretical AI, 9(20-3), 1997, pages 293-321.
- "Mathematical Model of Foraging in a Group of Robots: Effect of Interference", by K. Lerman and A. Galstyan, Autonomous Robots, 13(2), 2002, pages 127-141.
- "A formal framework for the study of task allocation in multi-robot systems", by B. Gerkey and M. Mataric´, International Journal of Robotics Research, 23(9):939-954, September 2004.
Project presentations: TBD
Project presentations: TBD
Project reports due! No late reports accepted.
- "Reinforcement Learning in the Multi-Robot Domain", Maja J Mataric´, Autonomous Robots, 4(1), pages 73-83, 1997.
- "A Self-Reconfigurable Modular Robot : Reconfiguration Planning and Experiments", by
Eiichi Yoshida, Satoshi Murata, Akiya Kamimura, Kohji Tomita, Haruhisa Kurokawa and Shigeru Kokaji, International Journal of Robotics Research, 21(10):903-916, October 2002.
- "Fast Replanning for Navigation in Unknown Terrain", by Sven Koenig and Maxim Likhachev, IEEE Transactions on Robotics , 21(3):354-364, June 2005.
The principles learned in this class will be applied in an individual
encouraged in this course. The simulation test-bed for the course is
Player/Stage/Gazebo, found at
Player is a general purpose language-independent network server for
Stage is a Player-compatible high-fidelity indoor multi-robot
Gazebo is a Player-compatible high-fidelity 3D outdoor simulation
testbed with dynamics. Other simulation test-beds are not acceptable.
Using Player/Stage/Gazebo allows for direct porting to
Player-compatible physical robots.
A Player-compatible ActivMedia Pioneer
1 AT (all terrain) robot will be made available as class resource for project
use. The robot is equipped with 7 sonar sensors and rugged locomotion. The
robot may be used only after a successful implementation of the project in
Stage/Gazebo first, which must be demonstrated to Prof. Mataric´ by Nov
Four Evolution Robotics ER1 platforms
will also be made available as class resources for project use. These
platforms are equipped with infra red sensors for local navigation, and a
camera. The ER1s are only Player-compatible in a limited sense (i.e., as a
position device) so please check the details of the player ER driver.
Finally, students who have existing access to physical robots through their
research may use those physical systems as their project test-beds. All
projects, software and robot use is contingent on project proposal approval.
Project proposals describing specific project goals,
implementation plan and platform, and evaluation metrics are due in
the middle of the semester, as stated in the syllabus
The following are the topics you should use as the basis for your project
proposal (you may think about adapt/combine these ideas when you write your
proposal, creativity is encouraged!)
- Intelligent assistive device:
Developing and integrating a weight sensor for an assistive robot. Involves
some limited hardware work, writing a Player driver, and testing on the Pioneer
- Imitation coach:
Extensions of existing arm movement evaluation software to develop an
interactive imitation training program. Involves graphics/animation rendering
of movement, use of motion capture equipment, and possibly use of the humanoid
torso robot or Bandit simulation.
- Vision-based face tracking:
Adapt Intel OpenCV or other existing face tracking software to work with
humanoid torso robot. Involves development of a tracking and filtering
code, we expect a Player driver as the interface for external users with
possible demonstration on a physical robot.
- Sound-based people tracking:
Interface the microphone array to use sound localization to detect and track
people's voices. Involves some device interfacing, Player device driver
writing, and testing.
- Speech Recognition in HRI:
One of the more popular speech recognition packages for HRI is sphinx2-0.4.
A number of important questions need to be experimentally studied and clearly
For example, comparing recognition quality under the following dimensions:
- software parameters and controls (internal sphinx arguments, size of vocabulary, volumes, etc.),
- environmental circumstances (background noise from crowds, sonars, etc.),
- different human subjects (w/ different accents).
- Real-time stereo vision:
Implement a stereo-vision software library for a humanoid robot
and demonstrate recognition on a stereo pair for a limited task.
- Humanoid Gestural Control:
Provide an interface that permits high-level gestural control for a mobile
- Mobile Greeter Robot:
Implement a mobile greeter robot that uses its spatial awareness to present
context-specific information, refer to various environmental features (possibly
other people sitting at predetermined desks) and direct a number of people
through an environment with an a priori map. (And/or you may focus on the
study of trade-offs on particular design variables, corralling behaviors, etc.)
Final projects will be presented to the class on
scheduled end-of-semester presentation days.
will be allotted 20 minutes, followed by
questions from the class. Live demos are encouraged but not
required.Helpful information for preparing your project presentation
can be found here
A detailed final paper
is a required part of the course project. A hard-copy
of the report is due on the last day of project presentations. The
report is to be made available on the Web as well. Helpful information
for preparing your project report can be found
An example of a successful project and project report can be found here.
examples of past projects and final papers, visit the previous
course Web page
Performance in the class will be based on participation in class
discussions, presentations of the readings, a quiz reviewing the basic
concepts covered in the class, a weekly 1/2 page summary report for
each of the papers, and a final project (with an associated paper and
demonstration). Regular attendance and participation is required; if
you need to miss a class, please speak to Prof. Mataric´ ahead of
Specifically, final course grades will consist of:
- 25% weekly summary reports
- 25% in-class discussion participation
- 25% in-class paper presentations
- 25% final project implementation and final paper
The following is a collection of other papers that are relevant to
this course, and/or have been used as readings in this course in the
past. To see the current reading list
course, look here
- "Symbol Grounding via a Hybrid Architecture in an Autonomous
Assembly System" by C. Malcolm and T. Smithers (1990)
- "Becoming Increasingly Reliable" by R. Simmons (1994)
- "Experiences with an Architecture for Intelligent, Reactive Agents" by
P. Bonasso, J. Firby, E. Gat, D. Kortenkamp, D. Miller and M. Slack (1996)
- "A Situated View of Representation and Control", Stanley Rosenschein
and Leslie Kaelbling, Artificial Intelligence, Special Issue on
Computational Research on Interaction and Agency, Elsevier Science,
pages 515-540, January/February 1995.
- "Internalized Plans: A Representation for Action Resources" by
D. Payton (1990)
- "Behavior-Based Control: Examples from Navigation, Learning, and Group
Behavior", Maja J Mataric´, Journal of Experimental and
Theoretical Artificial Intelligence, special issue on Software
Architectures for Physical Agents, H. Hexmoor, I. Horswill, and
D. Kortenkamp (eds.), 9(2-3), pages 323-336, 1997.
- "DAMN: A Distributed Architecture for Mobile Navigation" by
- "Situated Agents Can Have Goals" by P. Maes (1990)
- "ALVINN: An Autonomous Land Vehicle in A Neural Network" by D. Pomerleau (1992)
- "Rapidly Adapting Artificial Neural Networks for Autonomous Navigation"
by D. Pomerleau (1992)
- "Motor Schema Based Navigation for a Mobile Robot: An Approach to Programming
by Behavior", Ron Arkin, Proceedings of the IEEE Conference on
Robotics and Automation (ICRA-87), Raleigh, NC, pages 264-271, March
31 - April 3, 1987.
- "Model-based Learning for Mobile Robot Navigation from the
Dynamical Systems Perspective" by J. Tani (1996)
- "Rapid, Safe, and Incremental Learning of Navigation Strategies" by
J. Millan (1996)
"Using a Layered Control Architecture to Alleviate Planning with
Incomplete Information", Pete Bonasso and David Kortenkamp,
Proceedings, AAAI Spring Symposium: Planning with Incomplete
Information for Robot Problems, Stanford, CA, AAAI Press, pages
1-4, March 1996.
- "Behavior Coordination Mechanisms: State-of-the-Art", Paolo
Pirjanian, Technical Report IRIS-99-375, Institute for Robotics and
Intelligent Systems, School of Engineering, University of Southern
California, October 1999.
- "Learning to use selective attention and short-term memory
in sequential tasks" by A. McCallum (1996)
- "RoboShepherd: Learning a Complex Behavior" by A. Schultz et al
- "Flocks, Herds, and Schools: A Distributed Behavioral Model", C. Reynolds,
Computer Graphics, 21(4), pages 25-34, 1987.
- "Artificial Fishes with Autonomous Locomotion: Perception, Behavior
and Learning in a Simulated Physical World", Demetri Terzopoulos,
X. Tu, and R. Grzeszczuk, Artificial Life, 1(4), pages 327-351,
- "Group Behaviors for Systems with Significant Dynamics", David Brogan
and Jessica Hodgins, Autonomous Robots 4(1), pages 137-153, 1997.
- "Group Behaviors for Systems with Significant Dynamics" by D. Brogan
and J. Hodgins (1997)
- "Multiagent Systems: A Survey from a Machine Learning Perspective"
Peter Stone and Manuela Veloso, Autonomous Robots, 8(3), pages 345-383, 2000.
- "From Local Actions to Global Tasks: Stigmergy and Collective Robotics"
by R. Beckers et al (1994)
- "Swarm-Made Architectures" by J. L. Deneubourg et al (1991)
- "Macroscopic Model of an Aggregation Experiment Using Embodied Agents
in Groups of Time-Varying Sizes", Proceedings, IEEE Conference
on Systems, Man and Cybernetics (SMC), Hammamet, Tunisian, October
2002, pages 250-255.
- "Market-Based Multi-Robot Planning in a Distributed Layered
Architecture", Dani Goldberg, V. Cicirello, M.B. Dias, Reid Simmons,
S. Smith, and Anthony Stentz, Multi-Robot Systems: From Swarms to
Intelligent Automata: Proceedings, International Workshop on
Multi-Robot Systems, Kluwer Academic Publishers, Vol. 2, 2003,
- "An architecture for distributed cooperative-planning in a
behavior-based multi-robot system" by D. Jung and A. Zelinsky
- "Decentralized Motion Planning for Multiple Mobile Robots: The
Cocktail Party Model" by V. Lumelsky and K. Harinarayan (1997)
- "A Framework for Vision Based Formation Control," A. K. Das,
R. Fierro, V. Kumar, J. P. Ostrowski, J. Spletzer, and C. J. Taylor
- "Agents That Learn from Other Competitive Agents" by M. Asada et al
- "Cooperative behavior acquisition for mobile robots in dynamically
changing real worlds via vision-based reinforcement learning and
development" by M. Asada, E. Uchibe and K. Hosoda (1999)
- "Spontaneous, short-term interaction with mobile robots", Schulte,
J., Rosenberg, C., Thrun, S., Proceedings, IEEE International
Conference on Robotics and Automation, 10-15 May 1999, 658-663.
- "Investigating Models of Social Development Using a Humanoid
Robot"Brian Scassellati, in Biorobotics, Barbara Webb and
Thomas Consi, eds., MIT Press, 2000.
- "A Biological Perspective on Autonomous Agent Design" by R. Beer (1990)
- "Robotic Experiments in Cricket Phonotaxis" by B. Webb (1994)
- "Why Better Robots Make it Harder" by T. Smithers (1994)
- "Noise and the Reality Gap: The Use of Simulation in Evolutionary
Robotics" by N. Jakobi et al (1997)
- A few relevant books:
- Mobile Robots, Inspiration to Implementation, Joe Jones & Anita Flynn,
A. K. Peters.
- Genetic Algorithms in Search, Optimisation, and Machine Learning,
Goldberg, Addison Wesley.
- Artificial Life, Chris Langton, Addison Wesley.