  This paper is concerned with functional learning by utilizing two-stage
sampled distribution regression. We study a multi-penalty regularization
algorithm for distribution regression under the framework of learning theory.
The algorithm aims at regressing to real valued outputs from probability
measures. The theoretical analysis on distribution regression is far from
maturity and quite challenging, since only second stage samples are observable
in practical setting. In the algorithm, to transform information from samples,
we embed the distributions to a reproducing kernel Hilbert space
$\mathcal{H}_K$ associated with Mercer kernel $K$ via mean embedding technique.
The main contribution of the paper is to present a novel multi-penalty
regularization algorithm to capture more features of distribution regression
and derive optimal learning rates for the algorithm. The work also derives
learning rates for distribution regression in the nonstandard setting
$f_{\rho}\notin\mathcal{H}_K$, which is not explored in existing literature.
Moreover, we propose a distribution regression-based distributed learning
algorithm to face large-scale data or information challenge. The optimal
learning rates are derived for the distributed learning algorithm. By providing
new algorithms and showing their learning rates, we improve the existing work
in different aspects in the literature.
