It's a question that comes up quite often, and much has been said about it already. But I figured I'd throw in my two cents.
The situation is this: you've got a computer with 4GB RAM, but Windows (or Linux, or MacOS, etc.) reports you have only 3.25GB (or some other number less than 4GB). Where did the rest of the RAM go? How can you use the full amount? The answer to that last question is unfortunately that you can't, not with a 32 bit version of Windows anyway.
Here's what's going on. To read and write information in memory a computer program must specify where in memory it wishes to do so. For this it uses an address (I'm leaving virtual memory out of this discussion because it's irrelevant and would only complicate matters, so in this article when I say address I mean a physical address, that refers to an actual byte on your real RAM chips). Ever since the introduction of the 386 these addresses have been 32 bits in size. This means that there are 2 to the power of 32, or 4294967296 possible addresses. Because the x86 architecture addresses individual bytes, this means a maximum of 4294967296 bytes, or 4GB, can be addressed. This is the physical address space.
Many devices like your video or sound card use memory mapping to allow software to communicate with those devices. What this means is that they use certain addresses don't actually refer to a byte in memory; instead writing to that address sends information to that device. To do this they reserve a portion of the physical address space. If your video card is using 512MB of that address space and you install 4GB of memory in the system, you would need 4.5GB of address space to address all that, which you don't have. So 512MB of physical RAM cannot be addressed (because those addresses are used by the video card) and thus cannot be used or even "seen" by the OS.
Note that it's not only the video card; most other devices do this as well. Using the System Information tool in Windows (Start, Run, msinfo32.exe) you can see all hardware devices that do this and what address ranges they use under the "Hardware Resources, Memory" option (for the oldies: notice how there are still some devices that map into the old "upper memory" region (the area between the first 640KB and 1MB of RAM)). The ranges are shown as hexadecimal numbers.
Most 32 bit processors support an extended addressing mode called PAE (Physical Address Extension) which allows it to use 36 bit addresses for a total of 64GB of address space. 64 bit CPUs can use 64 bit addresses which allow for (theoretically) more than 16 million GB RAM (16 ExaBytes). However, for compatibility reasons all hardware devices will map into the first 4GB reason of address space (otherwise they wouldn't be able to run a 32 bit OS). So there's still an overlap and in the default scenario you'd still lose part of your RAM (even if you have e.g. 8GB RAM, you'd lose part of the first 4GB and see only e.g. 7.2GB). But most modern motherboards are capable of remapping the upper portion of RAM into the range above 4GB so all of it can theoretically be used (unless you actually have 64GB of RAM on a system with PAE).
I say theoretically because you still need OS support for the 36 bit addressing (or a real 64 bit OS on 64 bit systems). Windows does support PAE (among other things it's needed for Data Execution Prevention in Windows XP SP2 and up) but unless you have Windows Server it doesn't actually allow you to use the address above the 4GB boundary so you still lose RAM even if your motherboard has this support. The 64 bit versions of Windows (even the client versions) allow you to use 18TB of RAM so naturally it can use the full 4GB if your motherboard can do the remapping.
So simply put, if you want to use 4GB RAM and not lose any of it, you need a 64 bit CPU and either Windows Server, a 64 bit version of Windows, or some other operating system that either supports PAE or 64 bit.