  In this article, we discuss two specific classes of models - Gaussian Mixture
Copula models and Mixture of Factor Analyzers - and the advantages of doing
inference with gradient descent using automatic differentiation. Gaussian
mixture models are a popular class of clustering methods, that offers a
principled statistical approach to clustering. However, the underlying
assumption, that every mixing component is normally distributed, can often be
too rigid for several real life datasets. In order to to relax the assumption
about the normality of mixing components, a new class of parametric mixture
models that are based on Copula functions - Gaussian Mixuture Copula Models
were introduced. Estimating the parameters of the proposed Gaussian Mixture
Copula Model (GMCM) through maximum likelihood has been intractable due to the
positive semi-positive-definite constraints on the variance-covariance
matrices. Previous attempts were limited to maximizing a proxy-likelihood which
can be maximized using EM algorithm. These existing methods, even though easier
to implement, does not guarantee any convergence nor monotonic increase of the
GMCM Likelihood. In this paper, we use automatic differentiation tools to
maximize the exact likelihood of GMCM, at the same time avoiding any constraint
equations or Lagrange multipliers. We show how our method leads a monotonic
increase in likelihood and converges to a (local) optimum value of likelihood.
  In this paper, we also show how Automatic Differentiation can be used for
inference with Mixture of Factor Analyzers and advantages of doing so. We also
discuss how this method also has all the properties such as monotonic increase
in likelihood and convergence to a local optimum.
  Note that our work is also applicable to special cases of these two models -
for e.g. Simple Copula models, Factor Analyzer model, etc.
