  In this paper we give a completely new approach to the problem of covariate
selection in linear regression. A covariate or a set of covariates is included
only if it is better in the sense of least squares than the same number of
Gaussian covariates consisting of i.i.d. $N(0,1)$ random variables. The
Gaussian P-value is defined as the probability that the Gaussian covariates are
better. It is given in terms of the Beta distribution, it is exact and it holds
for all data. The covariate selection procedures based on this require only a
cut-off value $\alpha$ for the Gaussian P-value: the default value in this
paper is $\alpha=0.01$. The resulting procedures are very simple, very fast, do
not overfit and require only least squares. In particular there is no
regularization parameter, no data splitting, no use of simulations, no
shrinkage and no post selection inference is required. The paper includes the
results of simulations, applications to real data sets and theorems on the
asymptotic behaviour under the standard linear model. Here the stepwise
procedure performs overwhelmingly better than any other procedure we are aware
of. An R-package {\it gausscov} is available.
