  The aim of this note is to state a couple of general results about the
properties of the penalized maximum likelihood estimators (pMLE) and of the
posterior distribution for parametric models in a non-asymptotic setup and for
possibly large or even infinite parameter dimension. We consider a special
class of stochastically linear smooth (SLS) models satisfying two major
conditions: the stochastic component of the log-likelihood is linear in the
model parameter, while the expected log-likelihood is a smooth function. The
main results simplify a lot if the expected log-likelihood is concave. For the
pMLE, we establish a number of finite sample bounds about its concentration and
large deviations as well as the Fisher and Wilks expansion. The later results
extend the classical asymptotic Fisher and Wilks Theorems about the MLE to the
non-asymptotic setup with large parameter dimension which can depend on the
sample size. For the posterior distribution, our main result states a Gaussian
approximation of the posterior which can be viewed as a finite sample analog of
the prominent Bernstein--von Mises Theorem. In all bounds, the remainder is
given explicitly and can be evaluated in terms of the effective sample size and
effective parameter dimension. The results are dimension and coordinate free.
In spite of generality, all the presented bounds are nearly sharp and the
classical asymptotic results can be obtained as simple corollaries. Interesting
cases of logit regression and of estimation of a log-density with smooth or
truncation priors are used to specify the results and to explain the main
notions.
