I have personally never done this. I don't understand why so many sites do, if you do your development on a development server why would you ever need to shut down your production site?

I have always wondered about this.

What are they doing during this time, what requires doing this?

Big kicker for anything with big scale is that if one is changing database schemas in some way, one typically has some big, nasty maintenance scripts to run.

Now, these might take a second or so to run with your development dataset. But when you start measuring data in terabytes and petabytes, even adding a single column to a table can take hours.

So no matter how quick and automated the deployment is, you've still got data maintenance issues to get through. If you plan really well, you can put up a read-only mirror of the site while you are undergoing the process, but for many sites read-only is pointless and thus not worth the effort.
