I am currently learning to use Git by reading Pro Git. Right now I'm learning about branching and tags. My question is when should I use a branch and when should I use a tag?

For example, say I create a branch for version 1.1 of a project. When I finish and release this version, should I leave the branch to mark the release version? Or should I add a tag? If I add a tag, should I delete the version branch (assuming that it is merged into master or some other branch)?

In short: Best practice is branch out, merge often and keep always in sync.
There are pretty clear conventions about keeping your code in a separate branches from master branch:

You are about to make an implementation of major or disruptive change
You are about to make some changes that might not be used
You want to experiment on something that you are not sure it will work
When you are told to branch out, others might have something they need to do in master

Rule of thumb is after branching out, you should keep in sync with the master branch. Because eventually you need to merge it back to master. In order to avoid a huge complicated mess of conflicts when merging back, you should commit often, merge often.
Good practices to follow
A successful Git branching model by Vincent Driessen has good suggestions. If this branching model appeals to you consider the flow extension to git. Others have commented about flow.
Tagging practices
As you already know, Git gives you commit identifiers like 1.0-2-g1ab3183 but those are not tags! Tagging is done with git tag, and the tags that are created using git tag are the base for the commit identifiers git describe creates.
In another words, in Git you don't tag branches. You are tagging commits. It is correct to say that tag is just an annotated pointer to a commit.
Lets look at practical example that demonstrated it,

                        /-- [v1.0]
                       v
---.---.---.---S---.---A     <-- master
                         \ 
                           \-.---B     <-- test

Let's commit 'S' be commit pointed by tag 'v1.0'.  This commit is both on branch 'master' and on branch 'test'.  If you run "git describe" on top of commit 'A' (top of 'master' branch) you would get something like v1.0-2-g9c116e9.  If you run "git describe" on top of commit 'B' ( aka the 'test' branch) you would get something like v1.0-2-g3f55e41 , that is the case with default git-describe configuration.  Note that this result is slightly different.  v1.0-2-g9c116e9 means that we are at commit with sortened SHA-1 id of 9c116e9, 2 commits after tag v1.0.  There is no tag v1.0-2!
If you want your tag to appear only on branch 'master', you can create new commit (e.g. only update default / fallback version information in GIT-VERSION-FILE) after branching point of 'test' branch.  If you tag commits on 'test' branch with e.g. 'v1.0.3` it would be visible only from 'test'.
References
I have found many, many, useful blogs and posts to learn from. However, the ones that are professionally illustrated are rare ones. Thus, I would like to recommend a post - A successful Git branching model by @nvie. I have borrowed his illustration :)

