  This thesis aims to study some of the mathematical challenges that arise in
the analysis of statistical sequential decision-making algorithms for
postoperative patients follow-up. Stochastic bandits (multiarmed, contextual)
model the learning of a sequence of actions (policy) by an agent in an
uncertain environment in order to maximise observed rewards. To learn optimal
policies, bandit algorithms have to balance the exploitation of current
knowledge and the exploration of uncertain actions. Such algorithms have
largely been studied and deployed in industrial applications with large
datasets, low-risk decisions and clear modelling assumptions, such as
clickthrough rate maximisation in online advertising. By contrast, digital
health recommendations call for a whole new paradigm of small samples,
risk-averse agents and complex, nonparametric modelling. To this end, we
developed new safe, anytime-valid concentration bounds, (Bregman, empirical
Chernoff), introduced a new framework for risk-aware contextual bandits (with
elicitable risk measures) and analysed a novel class of nonparametric bandit
algorithms under weak assumptions (Dirichlet sampling). In addition to the
theoretical guarantees, these results are supported by in-depth empirical
evidence. Finally, as a first step towards personalised postoperative follow-up
recommendations, we developed with medical doctors and surgeons an
interpretable machine learning model to predict the long-term weight
trajectories of patients after bariatric surgery.
