I KNOW what moments are and how to calculate them and how to use the moment generating function for getting higher order moments. Yes, I know the math.
Now that I need to get my statistics knowledge lubricated for work, I thought I might as well ask this question – it's been nagging me for about a few years and back in college no professor knew the answer or would just dismiss the question (honestly).
So what does the word "moment" mean in this case? Why this choice of word? It doesn't sound intuitive to me (or I never heard it that way back in college :) Come to think of it I am equally curious with its usage in "moment of inertia" ;) but let's not focus on that for now.
So what does a "moment" of a distribution mean and what does it seek to do and why THAT word! :) Why does any one care about moments? At this moment I am feeling otherwise about that moment ;)
PS: Yes, I've probably asked a similar question on variance but I do value intuitive understanding over 'look in the book to find out' :)

According to the paper "First (?) Occurrence of Common Terms in Mathematical Statistics" by H.A. David, the first use of the word 'moment' in this situation was in a 1893 letter to Nature by Karl Pearson entitled "Asymmetrical Frequency Curves".

Neyman's 1938 Biometrika paper "A Historical Note on Karl Pearson's Deduction of the Moments of the Binomial" gives a good synopsis of the letter and Pearson's subsequent work on moments of the binomial distribution and the method of moments.  It's a really good read.  Hopefully you have access JSTOR for I don't have the time now to give a good summary of the paper (though I will this weekend).  Though I will mention one piece that may give insight as to why the term 'moment' was used.  From Neyman's paper:


  It [Pearson's memoir] deals primarily with methods of approximating 
      continuous frequency curves by means of some processes involving the 
      calculation of easy formulae.  One of these formulae considered was the 
      "point-binomial" or the "binomial with loaded ordinates".  The formula
      differs from what to-day we call a binomial, viz. (4), only by a factor 
      $\alpha$, representing the area under the continuous curve which it is desired
      to fit.


This is what eventually led to the 'method of moments.' Neyman goes over the Pearson's derivation of the binomial moments in the above paper.

And from Pearson's letter:


  We shall now proceed to find the first four moments of the system of 
      rectangles round GN. If the inertia of each rectangle might be considered 
      as concentrated along its mid vertical, we should have for the $s^{\text{th}}$ moment
      round NG, writing $d = c(1 + nq)$.


This hints at the fact that Pearson used the term 'moment' as an allusion to 'moment of inertia,' a term common in physics.

Here's a scan of most of Pearson's Nature letter:





You can view the entire article on page 615 here.
