Why do most browsers store browsing history by default? It seems like people often have to go to the hassle of manually deleting their history or using incognito mode. Are there some major advantages to storing this data in most cases?
There are a lot of advantages. Here are some:


Auto-completion of previously visited URLs you forgot, which can speed up the web surfing process tremendously. You might have remembered parts of a URL or website title, and your browser can usually pick those up if you typed them in. I love this feature.


This can offer extra security. As mentioned by kasperd, it can greatly reduce the risk of typosquatting. 

Storing previously-loaded data in a cache to speed up web-browsing. Great for slow connections. Great for reducing load on web servers.
Storing cookies so websites remember your login information, etc.

