Networks rock, especially electronic ones. In the history of the world there has been nothing like them–telegraph, telephone, cellular mobile, broadcast radio, cable television, Internet–to democratize the global, political economy, to bring us prosperity and freedom.
Astonishing as they are, electronic networks can’t hold a synaptic spark to the human brain. We know this intuitively–that the intelligence from the brain’s network of neurons makes even the most advanced supercomputers look pitifully stupid. And yet, this paradox: Why is it that a comparison of the components goes the other way? When you stack neurons up against modern transistors in switching speed, the brain looks pathetic.
Let me make a stab at resolving this brain-supercomputer contradiction by bringing up the network effect, as it is quantified in a 25-year-old, and still controversial, law that has my name attached.
The network effect, according to Wikipedia, “causes a good or service to have a value to a potential customer which depends on the number of other customers who own the good or are users of the service.” In the early 1980s I used an early description of this phenomenon to sell my invention, Ethernet, via
Using a 35mm slide (see chart below), I argued that my customers needed their Ethernets to grow above a certain critical mass if they were to reap the benefits of the network effect. 3Com sold $1,000 cards that connected desktop computers into a network. Here was the payoff: The cost of installing the cards at, say, a corporation would be proportional to the number of cards installed. The value of the network, though, would be proportional to the square of the number of users. Multiply the number of networked computers by ten and your systemwide cost goes up by a factor of ten but the value goes up a hundredfold.
Why should that be so? The network effect says that the value of that Ethernet card to the person on whose desk it sits is proportional to the number, N, of other computer users he can connect to. Now multiply this value by the number of users, and you have a value for the whole operation that is roughly proportional to N2.
I was a little vague about what “value” was, but in those days it had something to do with sharing expensive disks and printers, exchanging electronic mail within buildings and, for a pioneering few, access to what would later be called the Internet. My 3Com customers believed me and bought more Ethernets, which proved to be more valuable, and so they told their friends and bought even more. Ethernet became a standard in 1982, and 3Com went public in 1984. And now Internet plumbers are adding more than 108 Ethernet switch ports (wall sockets) per year–over a quarter-billion last year.
Now here’s some earlier history. Decades before Ethernet, IBM’s Herbert Grosch wrote that the performance of computers grew as the square of their cost. Grosch’s Law: Bigger computers are better. If you want to design an airplane wing or predict the weather, put a lot of iron in one place. In 1965
Recall that there is a critical mass beyond which the value of the network exceeds its cost. Where is this crossover point? You can find it by solving CxN=BxN2, where C is the constant of proportionality of cost and B is the constant of proportionality of value. The critical mass threshold can be expressed as N=C÷B. Not surprisingly, the lower the cost per connection, the lower the critical mass. The higher the value per connection, the lower the critical mass.
By Moore’s Law, C goes down by a factor of two every two years. And then there’s B, the value to each user of the connection to each other user. There are ever more values to connectivity because there are more things to do with it–
It was inevitable that Metcalfe’s Law would be taken in vain, as when it was invoked by Al Gore (1996) and Bill Clinton (1998) in commencement addresses at MIT. They (and I) used my law to inflate the Internet bubble, which burst in 2001. For those who believe the implosion was the worst thing that ever happened, my law is considered dangerous: They think that it overestimates the value of networks–and may even trigger Internet Bubble 2.0. I say, bring it on.
And bring on modern supercomputers. I am a director of a scientific supercomputer firm, SiCortex, in Maynard, Mass. Chief scientist Jud Leonard told me about his latest transistors and the cluster supercomputer these transistors are networked to produce. The SC5832 will cost less than $3 million and promises to increase scientific computation by factors of ten, per dollar, watt and foot. Its transistors are networked to form 64-bit arithmetic processors, six of which are fit on a “multicore” chip containing, thanks to Moore’s Law, 108 transistors. An SC5832 cabinet consumes only 20 kilowatts but holds 972 of Leonard’s multicore chips, or a total of 5,832 processors (hence the name), or 1011 transistors. Each of Leonard’s transistors can turn on or off 1010 times per second, but they are clocked at 500 megahertz to deliver, say, a mere billion computations per second.
Now, let’s turn to the brain’s “transistors.” It’s been a while since I studied neurons for my MIT undergraduate thesis in 1968 (fortunately we can’t find copies of that anymore), and so I consulted with Raymond Kurzweil, a pioneer in artificial intelligence, Tomaso Poggio, a professor at the McGovern Institute for Brain Research at MIT, and with Chris Diorio, a professor of computing at the University of Washington.
In his latest book, The Singularity Is Near (Viking Adult, 2005), Kurzweil shows that transistors have come quite a long way since 1968. Neurons, on the other hand, are still pretty much the same. Intelligent design is way faster than evolution. Poggio and his students are now successfully using transistors to emulate neuron networks, in particular those in the visual cortex of the brain. His computer software is beginning to approach the performance of the brain in picking out objects in a simple scene. Diorio studies how nervous tissues (i.e., brains) work in order to improve designs for computers. His research has resulted in some early “synaptic transistors” that learn from their inputs.
Kurzweil and Poggio gave me some very round numbers. They are not responsible for the fun I’m about to have with them.
The brain does its intelligent computing with 1011 neurons, interconnected in a very complicated network of roughly 1016 electrochemical connections, called synapses. Neuron synapses tick along relatively slowly, at about 103 times per second.
So, the brain and SC5832 both have 1011 computing elements. But the transistors run 106 times faster than the neurons. We might therefore easily assume, using round numbers and neglecting the network effect, that the SC5832′s network of transistors is 106 times as intelligent as the brain’s collection of neurons.
Not so. Kurzweil estimates that the human brain’s networked intelligence produces the equivalent of 1016 computations per second. This is 104 times more than the computational power of the SC5832. In other words, the brain is 106x104, or 1010, times smarter than it should be, just counting computing elements and multiplying their speeds.
Could it be that the network effect among synaptically networked neurons accounts for the brain’s surprising intelligence advantage over the SC5832? Is it too much to hope that Metcalfe’s Law might apply?
Some neuroscientists will rush to point out that I have failed to consider, among other things, that the brain is mostly not neurons. The brain also has 1012 glial cells. And while glia are hot in neuroscience, it is still thought that they don’t do much computing or communication, serving mainly to glue, feed, insulate and recycle the neurons they surround. It’s unlikely they account for much of any network effect among neurons.
On the other hand, an SC5832 can be configured with up to 8 terabytes of semiconductor memory chips. But these 1014 transistors don’t compute or connect much. It’s hard to see how adding them would help explain why the brain is so much smarter than the computer.
So, sidelining glial cells and memory transistors, what is the network effect in the brain when it goes up against SC5832?
Each transistor in SC5832 processor chips connects directly to one, two or maybe three other transistors. By contrast, each neuron, with its incoming dendrites and branching axons, forms 104 to 105 electrochemical synapse connections with adjacent neurons. Estimates are that the brain of a child has 1016 synapses (alas, the number goes down with age). So, the connections among transistors in the SC5832 are on the order of 1011, while connections among neurons in the brain number 1016.
But more connections mean greater value–in this case, intelligence. How much greater is the whole than the sum of its connected parts? The brain ends up not 106 less intelligent than the SC5832, as we were tempted to think, but 104 more intelligent. This is the network effect, surprising us by a factor of 1010.
How might we have predicted such a large effect? Metcalfe’s Law predicts that value is roughly proportional to N2. Try squaring the connection advantage of the brain over the SC5832, which is 105, to get 1010. Done.
Not so fast, says Professor Diorio, who notes that I’m squaring the wrong number. Metcalfe’s Law says you square the number of computing nodes, not the number of connections. Damn.
On the other hand, continues Diorio more kindly, it’s way too simple to say that neurons compute and synapses connect. It’s beginning to look like a lot of the brain’s intelligence is being computed not just along its 1011 neurons, but also at its 1016 synapses. This means that the brain really has 105 more networked computing nodes than the SC5832. Applying my law’s N2, which is widely considered something of an overestimate of the network effect, the brain should be 1010 times more intelligent than the SC5832, instead of 104 times it seems to be. But this seeming overestimate is easily accounted for by the fact that transistors are 106 times as fast as neurons, so that 106 times 104 equals 1010. Bingo: Metcalfe’s Law again exonerated.
There is one more level to human networking. Think of networking not just synapses but entire human brains. There are 6.6×109 of us, and we are rapidly getting better networked all the time. The Earth is a growing, “network-effect” metabrain.
Over the last 30 years, using Moore’s Law transistors and Metcalfe’s Law networks, we have gone from zero to 109 people on the Internet. With broadband deployment, the quality of an Internet connection is going up while costs go down. Social networking is proliferating and evolving. New collaboration modes are disrupting science, media and politics–for the better, I think.
The network effect is expanding the collective intelligence of the human race. We can hope that on the whole we humans are getting smarter by the square of some very large numbers. It’s enough to make one optimistic.
Robert M. Metcalfe invented Ethernet, founded 3Com and is a partner at Polaris Venture Partners.