  Since their emergence in the 1990's, the support vector machine and the
AdaBoost algorithm have spawned a wave of research in statistical machine
learning. Much of this new research falls into one of two broad categories:
kernel methods and ensemble methods. In this expository article, I discuss the
main ideas behind these two types of methods, namely how to transform linear
algorithms into nonlinear ones by using kernel functions, and how to make
predictions with an ensemble or a collection of models rather than a single
model. I also share my personal perspectives on how these ideas have influenced
and shaped my own research. In particular, I present two recent algorithms that
I have invented with my collaborators: LAGO, a fast kernel algorithm for
unbalanced classification and rare target detection; and Darwinian evolution in
parallel universes, an ensemble method for variable selection.
