  Knowing the uncertainty in a prediction is critical when making expensive
investment decisions and when patient safety is paramount, but machine learning
(ML) models in drug discovery typically provide only a single best estimate and
ignore all sources of uncertainty. Predictions from these models may therefore
be over-confident, which can put patients at risk and waste resources when
compounds that are destined to fail are further developed. Probabilistic
predictive models (PPMs) can incorporate uncertainty in both the data and
model, and return a distribution of predicted values that represents the
uncertainty in the prediction. PPMs not only let users know when predictions
are uncertain, but the intuitive output from these models makes communicating
risk easier and decision making better. Many popular machine learning methods
have a PPM or Bayesian analogue, making PPMs easy to fit into current
workflows. We use toxicity prediction as a running example, but the same
principles apply for all prediction models used in drug discovery. The
consequences of ignoring uncertainty and how PPMs account for uncertainty are
also described. We aim to make the discussion accessible to a broad
non-mathematical audience. Equations are provided to make ideas concrete for
mathematical readers (but can be skipped without loss of understanding) and
code is available for computational researchers
(https://github.com/stanlazic/ML_uncertainty_quantification).
