  With view to applications in stochastic analysis and geometry, we introduce a
new correspondence for positive definite kernels (p.d.) $K$ and their
associated reproducing kernel Hilbert spaces. With this we establish two kinds
of factorizations: (i) Probabilistic: Starting with a positive definite kernel
$K$ we analyze associated Gaussian processes $V$. Properties of the Gaussian
processes will be derived from certain factorizations of $K$, arising as a
covariance kernel of $V$. (ii) Geometric analysis: We discuss families of
measure spaces arising as boundaries for $K$. Our results entail an analysis of
a partial order on families of p.d. kernels, a duality for operators and
frames, optimization, Karhunen--Lo\`eve expansions, and factorizations.
Applications include a new boundary analysis for the Drury-Arveson kernel, and
for certain fractals arising as iterated function systems; and an
identification of optimal feature spaces in machine learning models.
