  We demonstrate that regression models can be estimated by working
independently in a row-wise fashion. We document a simple procedure which
allows for a wide class of econometric estimators to be implemented
cumulatively, where, in the limit, estimators can be produced without ever
storing more than a single line of data in a computer's memory. This result is
useful in understanding the mechanics of many common regression models. These
procedures can be used to speed up the computation of estimates computed via
OLS, IV, Ridge regression, LASSO, Elastic Net, and Non-linear models including
probit and logit, with all common modes of inference. This has implications for
estimation and inference with `big data', where memory constraints may imply
that working with all data at once is particularly costly. We additionally show
that even with moderately sized datasets, this method can reduce computation
time compared with traditional estimation routines.
