  We present a neural-symbolic framework for observing the environment and
continuously learning visual semantics and intuitive physics to reproduce them
in an interactive simulation. The framework consists of five parts, a
neural-symbolic hybrid network based on capsules for inverse graphics, an
episodic memory to store observations, an interaction network for intuitive
physics, a meta-learning agent that continuously improves the framework and a
querying language that acts as the framework's interface for simulation. By
means of lifelong meta-learning, the capsule network is expanded and trained
continuously, in order to better adapt to its environment with each iteration.
This enables it to learn new semantics using a few-shot approach and with
minimal input from an oracle over its lifetime. From what it learned through
observation, the part for intuitive physics infers all the required physical
properties of the objects in a scene, enabling predictions. Finally, a custom
query language ties all parts together, which allows to perform various mental
simulation tasks, such as navigation, sorting and simulation of a game
environment, with which we illustrate the potential of our novel approach.
