  Since many critical decisions impacting human lives are increasingly being
made by algorithms, it is important to ensure that the treatment of individuals
under such algorithms is demonstrably fair under reasonable notions of
fairness. One compelling notion proposed in the literature is that of
individual fairness (IF), which advocates that similar individuals should be
treated similarly (Dwork et al. 2012). Originally proposed for offline
decisions, this notion does not, however, account for temporal considerations
relevant for online decision-making. In this paper, we extend the notion of IF
to account for the time at which a decision is made, in settings where there
exists a notion of conduciveness of decisions as perceived by the affected
individuals. We introduce two definitions: (i) fairness-across-time (FT) and
(ii) fairness-in-hindsight (FH). FT is the simplest temporal extension of IF
where treatment of individuals is required to be individually fair relative to
the past as well as future, while in FH, we require a one-sided notion of
individual fairness that is defined relative to only the past decisions. We
show that these two definitions can have drastically different implications in
the setting where the principal needs to learn the utility model. Linear regret
relative to optimal individually fair decisions is inevitable under FT for
non-trivial examples. On the other hand, we design a new algorithm: Cautious
Fair Exploration (CaFE), which satisfies FH and achieves sub-linear regret
guarantees for a broad range of settings. We characterize lower bounds showing
that these guarantees are order-optimal in the worst case. FH can thus be
embedded as a primary safeguard against unfair discrimination in algorithmic
deployments, without hindering the ability to take good decisions in the
long-run.
