  In this paper we introduce a knowledge engine, which learns and shares
knowledge representations, for robots to carry out a variety of tasks. Building
such an engine brings with it the challenge of dealing with multiple data
modalities including symbols, natural language, haptic senses, robot
trajectories, visual features and many others. The \textit{knowledge} stored in
the engine comes from multiple sources including physical interactions that
robots have while performing tasks (perception, planning and control),
knowledge bases from the Internet and learned representations from several
robotics research groups.
  We discuss various technical aspects and associated challenges such as
modeling the correctness of knowledge, inferring latent information and
formulating different robotic tasks as queries to the knowledge engine. We
describe the system architecture and how it supports different mechanisms for
users and robots to interact with the engine. Finally, we demonstrate its use
in three important research areas: grounding natural language, perception, and
planning, which are the key building blocks for many robotic tasks. This
knowledge engine is a collaborative effort and we call it RoboBrain.
