  We address the multiple testing problem under the assumption that the
true/false hypotheses are driven by a Hidden Markov Model (HMM), which is
recognized as a fundamental setting to model multiple testing under dependence
since the seminal work of \citet{sun2009large}. While previous work has
concentrated on deriving specific procedures with a controlled False Discovery
Rate (FDR) under this model, following a recent trend in selective inference,
we consider the problem of establishing confidence bounds on the false
discovery proportion (FDP), for a user-selected set of hypotheses that can
depend on the observed data in an arbitrary way. We develop a methodology to
construct such confidence bounds first when the HMM model is known, then when
its parameters are unknown and estimated, including the data distribution under
the null and the alternative, using a nonparametric approach. In the latter
case, we propose a bootstrap-based methodology to take into account the effect
of parameter estimation error. We show that taking advantage of the assumed HMM
structure allows for a substantial improvement of confidence bound sharpness
over existing agnostic (structure-free) methods, as witnessed both via
numerical experiments and real data examples.
