  Decisions made by machine learning systems have increasing influence on the
world, yet it is common for machine learning algorithms to assume that no such
influence exists. An example is the use of the i.i.d. assumption in content
recommendation. In fact, the (choice of) content displayed can change users'
perceptions and preferences, or even drive them away, causing a shift in the
distribution of users. We introduce the term auto-induced distributional shift
(ADS) to describe the phenomenon of an algorithm causing a change in the
distribution of its own inputs. Our goal is to ensure that machine learning
systems do not leverage ADS to increase performance when doing so could be
undesirable. We demonstrate that changes to the learning algorithm, such as the
introduction of meta-learning, can cause hidden incentives for auto-induced
distributional shift (HI-ADS) to be revealed. To address this issue, we
introduce `unit tests' and a mitigation strategy for HI-ADS, as well as a toy
environment for modelling real-world issues with HI-ADS in content
recommendation, where we demonstrate that strong meta-learners achieve gains in
performance via ADS. We show meta-learning and Q-learning both sometimes fail
unit tests, but pass when using our mitigation strategy.
