  Convolved Gaussian Process (CGP) is able to capture the correlations not only
between inputs and outputs but also among the outputs. This allows a superior
performance of using CGP than standard Gaussian Process (GP) in the modelling
of Multiple-Input Multiple-Output (MIMO) systems when observations are missing
for some of outputs. Similar to standard GP, a key issue of CGP is the learning
of hyperparameters from a set of input-output observations. It typically
performed by maximizing the Log-Likelihood (LL) function which leads to an
unconstrained nonlinear and non-convex optimization problem. Algorithms such as
Conjugate Gradient (CG) or Broyden-Fletcher-Goldfarb-Shanno (BFGS) are commonly
used but they often get stuck in local optima, especially for CGP where there
are more hyperparameters. In addition, the LL value is not a reliable indicator
for judging the quality intermediate models in the optimization process. In
this paper, we propose to use enhanced Particle Swarm Optimization (PSO)
algorithms to solve this problem by minimizing the model output error instead.
This optimization criterion enables the quality of intermediate solutions to be
directly observable during the optimization process. Two enhancements to the
standard PSO algorithm which make use of gradient information and the multi-
start technique are proposed. Simulation results on the modelling of both
linear and nonlinear systems demonstrate the effectiveness of minimizing the
model output error to learn hyperparameters and the performance of using
enhanced algorithms.
