  Revealing hidden patterns in astronomical data is often the path to
fundamental scientific breakthroughs; meanwhile the complexity of scientific
inquiry increases as more subtle relationships are sought. Contemporary data
analysis problems often elude the capabilities of classical statistical
techniques, suggesting the use of cutting edge statistical methods. In this
light, astronomers have overlooked a whole family of statistical techniques for
exploratory data analysis and robust regression, the so-called Generalized
Linear Models (GLMs). In this paper -- the first in a series aimed at
illustrating the power of these methods in astronomical applications -- we
elucidate the potential of a particular class of GLMs for handling
binary/binomial data, the so-called logit and probit regression techniques,
from both a maximum likelihood and a Bayesian perspective. As a case in point,
we present the use of these GLMs to explore the conditions of star formation
activity and metal enrichment in primordial minihaloes from cosmological
hydro-simulations including detailed chemistry, gas physics, and stellar
feedback. We predict that for a dark mini-halo with metallicity $\approx 1.3
\times 10^{-4} Z_{\bigodot}$, an increase of $1.2 \times 10^{-2}$ in the gas
molecular fraction, increases the probability of star formation occurrence by a
factor of 75%. Finally, we highlight the use of receiver operating
characteristic curves as a diagnostic for binary classifiers, and ultimately we
use these to demonstrate the competitive predictive performance of GLMs against
the popular technique of artificial neural networks.
