  We present a methodology for using unlabeled data to design semi supervised
learning (SSL) methods that improve the prediction performance of supervised
learning for regression tasks. The main idea is to design different mechanisms
for integrating the unlabeled data, and include in each of them a mixing
parameter $\alpha$, controlling the weight given to the unlabeled data.
Focusing on Generalized Linear Models (GLM) and linear interpolators classes of
models, we analyze the characteristics of different mixing mechanisms, and
prove that in all cases, it is invariably beneficial to integrate the unlabeled
data with some nonzero mixing ratio $\alpha>0$, in terms of predictive
performance. Moreover, we provide a rigorous framework to estimate the best
mixing ratio $\alpha^*$ where mixed SSL delivers the best predictive
performance, while using the labeled and unlabeled data on hand.
  The effectiveness of our methodology in delivering substantial improvement
compared to the standard supervised models, in a variety of settings, is
demonstrated empirically through extensive simulation, in a manner that
supports the theoretical analysis. We also demonstrate the applicability of our
methodology (with some intuitive modifications) to improve more complex models,
such as deep neural networks, in real-world regression tasks.
