  Let $X_{1}=(W_{1},Y_{1}),\ldots,X_{n}=(W_{n},Y_{n})$ be $n$ pairs of
independent random variables. We assume that, for each $i\in\{1,\ldots,n\}$,
the conditional distribution of $Y_{i}$ given $W_{i}$ belongs to a
one-parameter exponential family with parameter
${\boldsymbol{\gamma}}^{\star}(W_{i})\in{\mathbb{R}}$, or at least, is close
enough to a distribution of this form. The objective of the present paper is to
estimate these conditional distributions on the basis of the observation
${\boldsymbol{X}}=(X_{1},\ldots,X_{n})$ and to do so, we propose a model
selection procedure together with a non-asymptotic risk bound for the resulted
estimator with respect to a Hellinger-type distance. When
${\boldsymbol{\gamma}}^{\star}$ does exist, the procedure allows to obtain an
estimator $\widehat{\boldsymbol{\gamma}}$ of ${\boldsymbol{\gamma}}^{\star}$
adapted to a wide range of the anisotropic Besov spaces. When
${\boldsymbol{\gamma}}^{\star}$ has a general additive or multiple index
structure, we construct suitable models and show the resulted estimators by our
procedure based on such models can circumvent the curse of dimensionality.
Moreover, we consider model selection problems for ReLU neural networks and
provide an example where estimation based on neural networks enjoys a much
faster converge rate than the classical models. Finally, we apply this
procedure to solve variable selection problem in exponential families. The
proofs in the paper rely on bounding the VC dimensions of several collections
of functions, which can be of independent interest.
