  Regression analysis is a central topic in statistical modeling, aiming to
estimate the relationships between a dependent variable, commonly referred to
as the response variable, and one or more independent variables, i.e.,
explanatory variables. Linear regression is by far the most popular method for
performing this task in several fields of research, such as prediction,
forecasting, or causal inference. Beyond various classical methods to solve
linear regression problems, such as Ordinary Least Squares, Ridge, or Lasso
regressions - which are often the foundation for more advanced machine learning
(ML) techniques - the latter have been successfully applied in this scenario
without a formal definition of statistical significance. At most, permutation
or classical analyses based on empirical measures (e.g., residuals or accuracy)
have been conducted to reflect the greater ability of ML estimations for
detection. In this paper, we introduce a method, named Statistical Agnostic
Regression (SAR), for evaluating the statistical significance of an ML-based
linear regression based on concentration inequalities of the actual risk using
the analysis of the worst case. To achieve this goal, similar to the
classification problem, we define a threshold to establish that there is
sufficient evidence with a probability of at least 1-eta to conclude that there
is a linear relationship in the population between the explanatory (feature)
and the response (label) variables. Simulations in only two dimensions
demonstrate the ability of the proposed agnostic test to provide a similar
analysis of variance given by the classical $F$ test for the slope parameter.
