  Datasets with extreme observations and/or heavy-tailed error distributions
are commonly encountered and should be analyzed with careful consideration of
these features from a statistical perspective. Small deviations from an assumed
model, such as the presence of outliers, can cause classical regression
procedures to break down, potentially leading to unreliable inferences. Other
distributional deviations, such as heteroscedasticity, can be handled by going
beyond the mean and modelling the scale parameter in terms of covariates. We
propose a method that accounts for heavy tails and heteroscedasticity through
the use of a generalized normal distribution (GND). The GND contains a
kurtosis-characterizing shape parameter that moves the model smoothly between
the normal distribution and the heavier-tailed Laplace distribution - thus
covering both classical and robust regression. A key component of statistical
inference is determining the set of covariates that influence the response
variable. While correctly accounting for kurtosis and heteroscedasticity is
crucial to this endeavour, a procedure for variable selection is still
required. For this purpose, we use a novel penalized estimation procedure that
avoids the typical computationally demanding grid search for tuning parameters.
This is particularly valuable in the distributional regression setting where
the location and scale parameters depend on covariates, since the standard
approach would have multiple tuning parameters (one for each distributional
parameter). We achieve this by using a "smooth information criterion" that can
be optimized directly, where the tuning parameters are fixed at log(n) in the
BIC case.
