  Finite linear least squares is one of the core problems of numerical linear
algebra, with countless applications across science and engineering.
Consequently, there is a rich and ongoing literature on algorithms for solving
linear least squares problems. In this paper, we explore a variant in which the
system's matrix has one infinite dimension (i.e., it is a quasimatrix). We call
such problems semi-infinite linear regression problems. As we show, the
semi-infinite case arises in several applications, such as supervised learning
and function approximation, and allows for novel interpretations of existing
algorithms. We explore semi-infinite linear regression rigorously and
algorithmically. To that end, we give a formal framework for working with
quasimatrices, and generalize several algorithms designed for the finite
problem to the infinite case. Finally, we suggest the use of various sampling
methods for obtaining an approximate solution.
