  The purpose of this article is to develop a general parametric estimation
theory that allows the derivation of the limit distribution of estimators in
non-regular models where the true parameter value may lie on the boundary of
the parameter space or where even identifiability fails. For that, we propose a
more general local approximation of the parameter space (at the true value)
than previous studies. This estimation theory is comprehensive in that it can
handle penalized estimation as well as quasi-maximum likelihood estimation
under such non-regular models. Besides, our results can apply to the so-called
non-ergodic statistics, where the Fisher information is random in the limit,
including the regular experiment that is locally asymptotically mixed normal.
In penalized estimation, depending on the boundary constraint, even the Bridge
estimator with $q<1$ does not necessarily give selection consistency.
Therefore, some sufficient condition for selection consistency is described,
precisely evaluating the balance between the boundary constraint and the form
of the penalty. Examples handled in the paper are: (i) ML estimation of the
generalized inverse Gaussian distribution, (ii) quasi-ML estimation of the
diffusion parameter in a non-ergodic It\^o process whose parameter space
consists of positive semi-definite symmetric matrices, while the drift
parameter is treated as nuisance and (iii) penalized ML estimation of variance
components of random effects in linear mixed models.
