Ed Felten quotes a summary of research that confirms the idiocy of this ranking formula:
Studies by U.S. Department of Education senior researcher Clifford Adelman in 1999 and 2005 showed that the best predictors of college graduation were not good high-school grades or test scores, but whether or not a student had an intense academic experience in high school. Such experiences were produced by taking higher-level math and English courses and struggling with the demands of college-level courses like AP or IB. Two recent studies looked at more than 150,000 students in California and Texas and found if they had passing scores on AP exams they were more likely to do well academically in college.
What is both amazing and depressing is that we wouldn't think of evaluating athletic performance with single tests. What matters is sustained performance over many events, which involves a wide range of physical and psychological processes. I became a decent skier (and a decent programmer) not by focusing on one test, but by working on harder and harder problems, and overcoming many failures, over a long period. Staying with it in the face of failure was in some ways much more important than scoring a perfect run (or writing a bug-free function first time) now and then. We all know that is what it takes. So why is our education so disconnected from it?
Maybe part of the problem is that education has become totally disconnected from doing. Most educational systems serve their own internal objectives, not the achievement of something of independent value.
All of the computer science I have learned — my college degree was in pure math — I learned because I was trying to some problem or understand someone else's solution. I would not have it any other way.
Even much of the math I learned in class only became alive when I had to remember it in solving some computational problem. I did pretty well in real and functional analysis, measure theory, differential geometry, abstract algebra, and mathematical logic in college, but all of that became much more real to me as I used it here and there in understanding problems in programming language design, formal linguistics, machine learning, and speech recognition.
The latest New Yorker has an interesting article on Gordon Bell. What struck me most there (leaving aside the questions of how to search a life's record) was the story of the young Gordon Bell learning by doing in his father's electrical shop. The formalization and bureaucratization of work has taken away most opportunities for effective apprenticeship. I doubt that we can fix our educational system before we create effective ways of learning by doing.
Learning by doing makes much more sense in one's internal economy. In standard education, you invest a lot of effort upfront in the hopes of some uncertain return in the distant future. In learning by doing, the return on investment is immediate: if I learn this piece of math or physics, I may be able to do this task and be rewarded (materially, psychologically, or socially) for my contribution to the project's success.
Learning by doing is also more efficient in individual resources, in that what we learn has an immediate use and is tailored to our interests, abilities, and needs. One-size-fits-all academic education is hugely wasteful in that it is based on fixed user-independent bundles of knowledge and skill. Once again, our processes and systems fail the long tail.