Understanding Pipelining and Superscalar Execution
Part II of Understanding the Microprocessor
(This feature for subscribers only!)
In my previous article, Understanding the
Microprocessor, I gave a high-level overview of what a microprocessor is and
how it functions. I talked about the kinds of tasks it performs and the
different steps that it goes through to carry out those tasks. The vision of the
microprocessor which emerged from that article was fairly simple and
straightforward. It was also fairly limited and primitive. Modern
microprocessors are more complex--they do more things in more complicated ways
than the first article really implies. The present article will discuss two
major innovations in processor design that have brought about huge leaps in
processor performance: pipelining and superscalar execution. If you're going to
understand modern processors, you have to master these two concepts. Most of the
newer tricks and techniques that I've covered in previous Ars articles on new
CPUs are aimed at improving the performance of one or both of these two
fundamental design approaches.
You'll often hear the term "pipelining" in discussions of CPU
technology, but the term itself is rarely defined. Pipelining is a fairly simple
concept, though, and the following section will make use of an analogy in order
to explain how it works.
(Note: The following discussion of pipelining is adapted from one of my articles on the K7, aka Athlon. I've reworked the analogy a bit, but the diagrams are the same.)
Let's say that we at Ars decided to quit doing tech journalism and go into the increasingly lucrative SUV manufacturing business. After some intense research, we determine that there are five stages in the SUV building process, as follows:
Stage 1: build the chassis.
Stage 2: drop the engine in the chassis.
Stage 3: put doors, a hood, and coverings on the chassis.
Stage 4: attach the wheels.
Stage 5: paint the SUV.
When we first hit on the idea of using an assembly line, we decided that it would be best to hire and train five crews of specialists, one for each stage of the SUV building process. There's one group to build the chassis, one to build the engine and drop it in, another for the wheels, etc. Each stage of the SUV building process takes a crew exactly one hour to complete.
Now, since we Ars guys are computer types and not industrial engineers, we're not too bright when it comes to making efficient use of factory resources. Also, because Ars was started back in the dot-com boom days, we're still kind of stuck in that mindset so we run a pretty chill shop with lots of free snacks, foosball tables, arcade games and other such employee perks. So our big plan is to have the factory run as follows: we line up all five crews in a row, and we have the first crew start an SUV at Stage 1. After Stage 1 is complete, the SUV moves down the line to the next stage and the next crew drops the engine in. While the Stage 2 Crew is installing the engine in the chassis that the Stage 1 Crew just built, the Stage 1 Crew (along with all of the rest of the crews) is free to go play foosball, watch the big-screen plasma TV in the
break room, surf the 'net, etc. Once the Stage 2 Crew is done, the SUV moves down to Stage 3 and the Stage 3 Crew takes over while the Stage 2 Crew hits the
break room to party with everyone else.
The SUV moves on down the line through all five stages this way, with only one crew working on one stage at any given time while the rest of the crews are idle. Once the completed SUV finishes Stage 5, the crew at Stage 1 then starts on another SUV. At this rate, it takes exactly five hours to finish a single SUV, and our factory puts out one SUV every five hours.
The following picture shows our assembly line with a SUV in Stage 2. It's having it's engine installed while the other crews are idle.
Now, you may be thinking, Why not just have one full-time crew to do all the work? The nominal, stated reason (i.e. the reason Caesar gave when he was pitching this assembly line idea to the rest of the Ars team) is that each stage of construction requires a specific skill set. If we hire five, highly skilled crews to do the job then it'll wind up taking us less time overall to build a SUV than if we hired only one crew that's not very good (or very fast) at completing any of the five stages.
Of course, the real reason for having five crews (at least as far as I'm concerned) is that Caesar, in classic Roman nepotistic fashion, believes in hiring his relatives and in paying them outrageously to spend most of their time playing foosball. And Caesar has a pretty large family. But for the purposes of the present discussion we'll pretend I didn't say this.