  Two linearly uncorrelated binary variables must be also independent because
non-linear dependence cannot manifest with only two possible states. This
inherent linearity is the atom of dependency constituting any complex form of
relationship. Inspired by this observation, we develop a framework called
binary expansion linear effect (BELIEF) for understanding arbitrary
relationships with a binary outcome. Models from the BELIEF framework are
easily interpretable because they describe the association of binary variables
in the language of linear models, yielding convenient theoretical insight and
striking Gaussian parallels. With BELIEF, one may study generalized linear
models (GLM) through transparent linear models, providing insight into how the
choice of link affects modeling. For example, setting a GLM interaction
coefficient to zero does not necessarily lead to the kind of no-interaction
model assumption as understood under their linear model counterparts.
Furthermore, for a binary response, maximum likelihood estimation for GLMs
paradoxically fails under complete separation, when the data are most
discriminative, whereas BELIEF estimation automatically reveals the perfect
predictor in the data that is responsible for complete separation. We explore
these phenomena and provide related theoretical results. We also provide
preliminary empirical demonstration of some theoretical results.
