  We introduce a semi-supervised learning estimator which tends to the first
kernel principal component as the number of labelled points vanishes. Our
approach is based on the notion of optimal target vector, which is defined as
follows. Given an input data-set of ${\bf x}$ values, the optimal target vector
$\mathbf{y}$ is such that treating it as the target and using kernel ridge
regression to model the dependency of $y$ on ${\bf x}$, the training error
achieves its minimum value. For an unlabeled data set, the first kernel
principal component is the optimal vector. In the case one is given a partially
labeled data set, still one may look for the optimal target vector minimizing
the training error. We use this new estimator in two directions. As a
substitute of kernel principal component analysis, in the case one has some
labeled data, to produce dimensionality reduction. Second, to develop a
semi-supervised regression and classification algorithm for transductive
inference. We show application of the proposed method in both directions.
