  Linear regression is perhaps one of the most popular statistical concepts,
which permeates almost every scientific field of study. Due to the technical
simplicity and wide applicability of linear regression, attention is almost
always quickly directed to the algorithmic or computational side of linear
regression. In particular, the underlying mathematics of stochastic linear
regression itself as an entity usually gets either a peripheral treatment or a
relatively in-depth but ad hoc treatment depending on the type of concerned
problems; in other words, compared to the extensiveness of the study of
mathematical properties of the "derivatives" of stochastic linear regression
such as the least squares estimator, the mathematics of stochastic linear
regression itself seems to have not yet received a due intrinsic treatment.
Apart from the conceptual importance, a consequence of an insufficient or
possibly inaccurate understanding of stochastic linear regression would be the
recurrence for the role of stochastic linear regression in the important (and
more sophisticated) context of structural equation modeling to be misperceived
or taught in a misleading way. We believe this pity is rectifiable when the
fundamental concepts are correctly classified. Accompanied by some
illustrative, distinguishing examples and counterexamples, we intend to pave
out the mathematical framework for stochastic linear regression, in a rigorous
but non-technical way, by giving new results and pasting together several
fundamental known results that are, we believe, both enlightening and
conceptually useful, and that had not yet been systematically documented in the
related literature. As a minor contribution, the way we arrange the fundamental
known results would be the first attempt in the related literature.
