  Three methods of least squares are examined for fitting a line to points in
the plane. Two well known methods are to minimize sums of squares of vertical
or horizontal distances to the line. Less known is to minimize sums of squares
of distances to the line. Concise proofs are given for each method using a
combination of the first derivative test for functions of two variables and
completing the square. The three methods are compared and the distances to the
line method appears to be favourable in most circumstances. They generally draw
different regression lines. The method of vertical displacements typically
gives a slope of too small magnitude while the method of horizontal
displacements typically gives a slope of too large magnitude. An inequality
involving the three slopes is proved. Rotating all the data points in the same
way with these two methods does not result in the regression line being rotated
the same way. However, the distance to the line method is invariant under
rotations.
