  Machines, not humans, are the world's dominant knowledge accumulators but
humans remain the dominant decision makers. Interpreting and disseminating the
knowledge accumulated by machines requires expertise, time, and is prone to
failure. The problem of how best to convey accumulated knowledge from computers
to humans is a critical bottleneck in the broader application of machine
learning. We propose an approach based on human teaching where the problem is
formalized as selecting a small subset of the data that will, with high
probability, lead the human user to the correct inference. This approach,
though successful for modeling human learning in simple laboratory experiments,
has failed to achieve broader relevance due to challenges in formulating
general and scalable algorithms. We propose general-purpose teaching via
pseudo-marginal sampling and demonstrate the algorithm by teaching topic
models. Simulation results show our sampling-based approach: effectively
approximates the probability where ground-truth is possible via enumeration,
results in data that are markedly different from those expected by random
sampling, and speeds learning especially for small amounts of data. Application
to movie synopsis data illustrates differences between teaching and random
sampling for teaching distributions and specific topics, and demonstrates gains
in scalability and applicability to real-world problems.
