  How we store information in our mind has been a major intriguing open
question. We approach this question not from a physiological standpoint as to
how information is physically stored in the brain, but from a conceptual and
algorithm standpoint as to the right data structures to be used to organize and
index information. Here we propose a memory architecture directly based on the
recursive sketching ideas from the paper "Recursive Sketches for Modular Deep
Networks", ICML 2019 (arXiv:1905.12730), to store information in memory as
concise sketches. We also give a high level, informal exposition of the
recursive sketching idea from the paper that makes use of subspace embeddings
to capture deep network computations into a concise sketch. These sketches form
an implicit knowledge graph that can be used to find related information via
sketches from the past while processing an event.
