  Information is physical, and for a physical theory to be universal, it should
model observers as physical systems, with concrete memories where they store
the information acquired through experiments and reasoning. Here we address
these issues in Spekkens' toy theory, a non-contextual epistemically restricted
model that partially mimics the behaviour of quantum mechanics. We propose a
way to model physical implementations of agents, memories, measurements,
conditional actions and information processing. We find that the actions of toy
agents are severely limited: although there are non-orthogonal states in the
theory, there is no way for physical agents to consciously prepare them. Their
memories are also constrained: agents cannot forget in which of two arbitrary
states a system is. Finally, we formalize the process of making inferences
about other agents' experiments and model multi-agent experiments like Wigner's
friend. Unlike quantum theory or box world, in the toy theory there are no
inconsistencies when physical agents reason about each other's knowledge.
