In particular, I am referring to the Pearson product-moment correlation coefficient.
What's the difference between the correlation between $X$ and $Y$ and a linear regression predicting $Y$ from $X$?
First, some similarities:

the standardised regression coefficient is the same as Pearson's correlation coefficient
The square of Pearson's correlation coefficient is the same as the $R^2$ in  simple linear regression
The sign of the unstandardized coefficient (i.e., whether it is positive or negative) will the same as the sign of the correlation coefficient.
Neither simple linear regression nor correlation answer questions of causality directly. This point is important, because I've met people that think that simple regression can magically allow an inference that $X$ causes $Y$.
Standard tests of the null hypothesis (i.e., "correlation = 0" or, equivalently, "slope = 0" for the regression in either order), such as carried out by lm and cor.test in R, will yield identical p-values.

Second, some differences:

The regression equation (i.e., $a + bX$) can be used to make predictions on $Y$ based on values of $X$
While correlation typically refers to the linear relationship, it can refer to other forms of dependence, such as polynomial or truly nonlinear relationships
While correlation typically refers to Pearson's correlation coefficient, there are other types of correlation, such as Spearman's.
The correlation between X and Y is the same as the correlation between Y and X. In contrast, the unstandardized coefficient typically changes when moving from a model predicting Y from X to a model predicting X from Y.

