  Deep learning models can extract predictive and actionable information from
complex inputs. The richer the inputs, the better these models usually perform.
However, models that leverage rich inputs (e.g., multi-modality) can be
difficult to deploy widely, because some inputs may be missing at inference.
Current popular solutions to this problem include marginalization, imputation,
and training multiple models. Marginalization can obtain calibrated predictions
but it is computationally costly and therefore only feasible for low
dimensional inputs. Imputation may result in inaccurate predictions because it
employs point estimates for missing variables and does not work well for high
dimensional inputs (e.g., images). Training multiple models whereby each model
takes different subsets of inputs can work well but requires knowing missing
input patterns in advance. Furthermore, training and retaining multiple models
can be costly. We propose an efficient way to learn both the conditional
distribution using full inputs and the marginal distributions. Our method,
Knockout, randomly replaces input features with appropriate placeholder values
during training. We provide a theoretical justification of Knockout and show
that it can be viewed as an implicit marginalization strategy. We evaluate
Knockout in a wide range of simulations and real-world datasets and show that
it can offer strong empirical performance.
