  Clinical machine learning applications are often plagued with confounders
that can impact the generalizability and predictive performance of the
learners. Confounding is especially problematic in remote digital health
studies where the participants self-select to enter the study, thereby making
it challenging to balance the demographic characteristics of participants. One
effective approach to combat confounding is to match samples with respect to
the confounding variables in order to balance the data. This procedure,
however, leads to smaller datasets and hence impact the inferences drawn from
the learners. Alternatively, confounding adjustment methods that make more
efficient use of the data (e.g., inverse probability weighting) usually rely on
modeling assumptions, and it is unclear how robust these methods are to
violations of these assumptions. Here, rather than proposing a new approach to
control for confounding, we develop novel permutation based statistical methods
to detect and quantify the influence of observed confounders, and estimate the
unconfounded performance of the learner. Our tools can be used to evaluate the
effectiveness of existing confounding adjustment methods. We illustrate their
application using real-life data from a Parkinson's disease mobile health study
collected in an uncontrolled environment.
