  We explore the probabilistic partition of unity network (PPOU-Net) model in
the context of high-dimensional regression problems and propose a general
framework focusing on adaptive dimensionality reduction. With the proposed
framework, the target function is approximated by a mixture of experts model on
a low-dimensional manifold, where each cluster is associated with a local
fixed-degree polynomial. We present a training strategy that leverages the
expectation maximization (EM) algorithm. During the training, we alternate
between (i) applying gradient descent to update the DNN coefficients; and (ii)
using closed-form formulae derived from the EM algorithm to update the mixture
of experts model parameters. Under the probabilistic formulation, step (ii)
admits the form of embarrassingly parallelizable weighted least-squares solves.
The PPOU-Nets consistently outperform the baseline fully-connected neural
networks of comparable sizes in numerical experiments of various data
dimensions. We also explore the proposed model in applications of quantum
computing, where the PPOU-Nets act as surrogate models for cost landscapes
associated with variational quantum circuits.
