  We consider the problem of iterative machine teaching, where a teacher
sequentially provides examples based on the status of a learner under a
discrete input space (i.e., a pool of finite samples), which greatly limits the
teacher's capability. To address this issue, we study iterative teaching under
a continuous input space where the input example (i.e., image) can be either
generated by solving an optimization problem or drawn directly from a
continuous distribution. Specifically, we propose data hallucination teaching
(DHT) where the teacher can generate input data intelligently based on labels,
the learner's status and the target concept. We study a number of challenging
teaching setups (e.g., linear/neural learners in omniscient and black-box
settings). Extensive empirical results verify the effectiveness of DHT.
