A quick Google search doesn't reveal whether it is important to logout of webapps (online banking, Amazon, Facebook, etc.), or if I am safe just closing the tab or browser. I am sure I heard on some TV show that it's best to logout...

What possible threats am I exposing myself to if I don't properly log out?

This is not a trivial, simplistic question. There are several different aspects you need to consider, and several different mechanisms and countermeasures that apply to several different threats in several different scenarios that are affected by several different clients. Let's examine these one at a time. (There will be a TL;DR at the end...)   


If you're using a Public computer: LOG OUT.
Any service you keep an account on should not be left logged in on a publicly accessible machine. 
If you're using a Trivial unsensitive service: STAY LOGGED IN.
This applies only to throw-away, temporary accounts, such as for Internet radio, where giving away access is nothing more than a nuisance.  
If you're using Public Wifi: LOG OUT.
Since the network is inherently untrusted, there is one big obvious THREAT: Session cookie theft. It is possible that your session was hijacked, and someone - either someone else on the network, or the hotspot itself - stole your session cookie. Of course, if this was the case, you wouldn't know, but then you may not be able to really log out either (if it is a malicious network or MITM, they have control of your entire connection - they might simply drop your logout request).

That said, 3rd party theft of just your session cookie IS a valid threat (e.g. FireSheep), and explicit logging out prevents unlimited use. (Basically the damage may have been done already, but this stops it from continuing.)

Even better would be to go to a trusted network, login, and explicitly log back out, just in case the MITM blocked your logout. Better yet is to change your password on the trusted site... But best to never access a non-trivial, sensitive site from an untrusted network. 
If you're using All-day applications: STAY LOGGED IN.
For services you use all day and want quick/easy access to, e.g. Facebook, email, etc - IF this is your own private (or work) computer on a trusted network, it is a sensible trade-off to leave your browser logged in long-term.

THREAT: Malicious bystander 

Lock your computer any time you step away, even to get a cup of coffee. Or lock your office, if you have a physical door that noone else can get through. (Or have a home office, wheee!) Periodically log out and back in. Monitor any posts "you" make.

THREAT: Other sites can register that you are logged in (e.g. to show you that important "Like" icon from Facebook). This is part of the trade-off that applies, while there are wider implications that are out of scope for this answer. 
If you are using Any application that uses HTTP Basic Authentication (e.g. many routers): LOG OUT, AND CLOSE ALL BROWSER WINDOWS. Here is where it gets interesting, and this applies to the next section too.    

When you log in to a webapp using Basic AuthN, the browser caches your password, and sends it on every request. The browser's BasicAuth mechanism has no concept of session. Even if you repeatedly logout, the webapp does not - neither serverside nor clientside - have any way of "killing" the session. The only way to clear those cached credentials, is to kill the browser process.

HOWEVER. Your choice of browser matters for this concept of "browser process". E.g.: 


Firefox: Always a single process, no matter how many tabs and windows you open. 
Chrome: Each tab is a separate process. However, there is another, "global", parent process. All the tab processes are child processes of this one (aka "job process" in Windows parlance), and they all share process memory through the parent. This is also true if you open a new window. So while Chrome's ample use of child processes with shared parent make its tabs particularly lively and robust, the downside is sharing process state. In other words, the only way to remove cached BasicAuth credentials from Chrome is to close all Chrome windows, every last one. Closing the tab alone will not help.   
IE: tab/process model is identical (or similar) to Chrome... with one exception. By default, IE also opens all tabs in a child of the parent process. (Actually, this is not 100% accurate - some tabs share a child process with other tabs - but this doesnt matter in reality). However, if you add "-NoFrameMerging" to the IE commandline, it will create a completely new IE parent process. The difference here is that you can e.g. create a new parent window just to login to your router, and then close just that window when you are done. This clears your BasicAuth cache, without touching any other open IE windows. (Side note: it is actually possible to do this with Chrome too! It is a lot more involved, though, and requires you to create another browser profile on your machine.) 

If you are using Sensitive applications, e.g. banking apps - ALWAYS EXPLICITLY LOG OUT, AND CLOSE ALL BROWSER WINDOWS. This part is a little more complicated, but a lot of the dependencies were already covered above.   

THREAT: Malicious bystander Locking your computer, as above, would make sense, however there is no need for the trade-off from before. Just log out.

Session Timeout: In addition, most sensitive (e.g. banking) apps should implement some form of automatic idle timeout, so if you go out for the afternoon your session will automatically die at some point. This might not help with this threat, since the malicious bystander may just hop on your computer if you step out for 4 1/2 minutes to refill your coffee. 

THREAT: Session cookie theft

Hopefully, sensitive apps are actively preventing this, with e.g. HTTPS, IDS, geo/fraud detection, etc. That said, it still makes sense to close that "window of opportunity", just in case - defense in depth, and all that.

Session Timeout: As before, most sensitive (e.g. banking) apps should implement some form of automatic idle timeout, and will help minimize this threat too. However, even if you do know for a fact that this app does implement idle timeouts correctly, there is still a window of opportunity for the attacker. That said, in a relatively-secure app this is not much of a threat.

THREAT: Cross Site Request Forgery (CSRF) 

This is the one you need to worry about.   

Say you are logged in to your bank. In the same window, in a different tab, you are browsing some dubious website. While viewing this website, it might be surreptitiously testing various well-known banksites, to see if you happen to be logged in to one of them. If you are, it will mount the CSRF attack (not all bank sites are vulnerable to this, but many still are). CSRF'd!    

Okay. Now say you are smarter than that other guy, and dont browse suspicious sites the same time your banksite is open. So, after you finish on your bank, you carefully close the tab. Only then do you open a new tab to browse to the dodgy site. Well, problem is, you are still logged in, and will be for a while (typically around 30 minutes, but it could be as little as 10 or as much as an hour...). CSRF'd!.   

(Note that the session timeout here does help, by shortening the window of opportunity, but there is still a chance of this happening within the window).      

Hmm. Well, I know, let's open a new browser window! Use that for bank work, then again CLOSE the tab, and again open a new one for the malware sites I like to play with. Whoops, see the above section on Basic Authentication - your choice of browser matters.

Unless you're using "incognito/private browsing", or the "-NoFrameMerging" flag for IE, you are still in the same process family, and this still-open session will be shared between all your windows, at least until the server hits the idle timeout. Assuming it hasn't already been co-opted. CSRF'd! 

Okay, one more, just one. I read this overly long post somewhere, about how I always need to logout from my sensitive apps - so I do just that, before popping on to my criminal sites. Unfortunately, the application "forgot" to do a proper logout, it just redirects me out of the application (or erases my cookie, or...) instead of invalidating it on the server... CSRF'd!




So, TL;DR? 


  
  If you care about your account on this site: LOG OUT.   
  If you care about your account, and it uses Basic Authentication: LOG OUT AND CLOSE ALL BROWSER TABS AND WINDOWS.   
  If you don't care about your account - it doesn't matter what you do, so stop asking :-).
  




P.S. I did not cover things like Flash cookies, non-http sessions, and Integrated Windows Authentication. Enough is enough. 
