  We study the problem of designing minimax procedures in linear regression
under the quantile risk. We start by considering the realizable setting with
independent Gaussian noise, where for any given noise level and distribution of
inputs, we obtain the exact minimax quantile risk for a rich family of error
functions and establish the minimaxity of OLS. This improves on the known lower
bounds for the special case of square error, and provides us with a lower bound
on the minimax quantile risk over larger sets of distributions. Under the
square error and a fourth moment assumption on the distribution of inputs, we
show that this lower bound is tight over a larger class of problems.
Specifically, we prove a matching upper bound on the worst-case quantile risk
of a variant of the recently proposed min-max regression procedure, thereby
establishing its minimaxity, up to absolute constants. We illustrate the
usefulness of our approach by extending this result to all $p$-th power error
functions for $p \in (2, \infty)$. Along the way, we develop a generic analogue
to the classical Bayesian method for lower bounding the minimax risk when
working with the quantile risk, as well as a tight characterization of the
quantiles of the smallest eigenvalue of the sample covariance matrix.
