Q: Why do I have to use free on a pointer but not a normal declaration? Why do I have to use free() when I declare a pointer such as:
int *temp = (int*)malloc(sizeof(int))
*temp = 3;

but not when I do:
int temp = 3;


A: Because the language let's you pick between the stack and the heap. 
Reasons why you'd want to pick between the stack and the heap:


*

*Variables on the heap purposely do not free themselves so that you can use them past the scope of your code block or function.

*It is more efficient to work with the stack compared to the heap

*On most compilers you cannot pick the size of an object or array on the stack at runtime, so the heap would be used here. 


Why the heap can't be automatically freed:
Because there is no way to know when you are done with the memory.  There are ways to emulate garbage collection, but this involves when you have no more variables on the stack and heap holding a pointer to the data on the heap. 
More on Stack vs Heap:
The C language let's you chose whether you want to define your variables on the stack or the heap.   


*

*Variables on the stack are automatically freed when they fall out of scope.

*Variables on the heap are not automatically freed.


malloc create variables on the heap.   A simple declaration such as int x; creates a variable on the stack. 
See further reading on stack vs heap in my answer here. 
Pointers:
Just to clarify: Pointer varaibles are created on the stack and they hold a memory address to the data allocated on the heap.  They take 4 bytes on the stack on a 32-bit system and 8 bytes on the stack on a 64-bit system. 

A: It should be noted that C has no concept of stack or heap, although the preceding answers are correct ~99% of the time and give great insight.
C defines three storage durations for objects: static, automatic and allocated. (§6.2.4.1)
Static objects (such as global variables) are available for the duration of an entire program.
Automatic objects exist as long as their variable is in scope. They cease to exist as soon as it goes out of scope.
Note that these are the two extremes. C gives you a point in between: Allocated objects. (The search term would be dynamically allocated memory.) With those, you tell the computer when objects should start and end their existence. And this is done by using the standard functions malloc() (or derivatives) and free().
Strictly speaking, you do not have to call free(). Or maybe you do (you'd have to read the standard for an authoritive point on this), but you could do it all at the end of main(), just before the program terminates. Or, leave it to the operating system to do it for you (which most, if not all, do.) But that would again be an extremity - objects come into existence when you call malloc(), go out when your program terminates.
I should not have to talk in detail about the practical implications here: Memory is finite. You can only allocate so many bytes before you run out of memory. Using static objects for everything would be too wasteful; trying to reuse chunks or one big chunk of static memory would be hard and in any case analogous to the dynamic allocation approach. Using automatic storage for long-lived objects would force you to make their scope as big as possible, roughly corresponding to those of static objects anyway.
-- 
Now, some notes:
{
    int *temp = malloc(sizeof(int));
    *temp = 5;
    //free(temp);
}

Note that temp here is an automatic object. It will only live as long as its scope, which ends at }. The object it's pointing to, however, is allocated. It will exist until you call free() on its address. Since temp contains the only copy of that address, you will lose the chance to call free() once temp goes out of scope. Some memory will be permanently allocated, yet unavailable. This is called a memory leak.
Garbage collection is another method for managing object storage. An implementation in C might look like:
{
    int *temp = gc_malloc(sizeof(int));
    *temp = 5;
}

Where at the }, the computer would decide that the last reference, temp, to the allocated object was lost and that it would be a good idea to free it.
It is a trade-off where you don't have to worry about free()ing objects (which is not a minor thing as simple examples might make you think), but where gc_malloc() here is more complex than a simple malloc(), and there's invisible code executing at that } where temp goes out of scope. And it is a whole different topic how the computer might decide that temp was the last reference. (Some practical solutions might involve you writing more code around "int *temp".)

A: Normal declarations are put on the stack.  When the function returns the stack pointer reverts to the value it had before the function was called, so the memory is automatically reclaimed.
Malloc-based declarations are allocated from the 'heap', which requires the programmer to manage allocations and deallocations.

A: Alnitak is correct.  I'd like to point out what "on the stack" really means.
When a program makes a function call, it expects the function to do some amount of work, then return and continue to the next line of code.  The function has no way of knowing where to return when the function is complete.  So, the machine has a call stack for each program used to push the address of the following statement in the program before calling the function.  The "return" statement simply pops the program address and jumps to it.
The stack is also a handy scratch pad of temporary space.  It is possible to write into unused areas of the stack.  Declaring a local variable inside a C function does exactly this.  When the function returns, the stack does not need to be cleaned up, freed, or otherwise processed because it was just a temporary space anyway and now goes out of scope.
In contrast, calling malloc() allocates memory from the heap, which explicitly sets aside memory for the program and remains in scope for as long as the program is running.  Thus, if you don't free() the memory, it will remain allocated and considered a memory leak.

A: The need to free() doesn't depend on whether or not you've declared a pointer, but rather whether or not you've malloc()ed memory.
Like Brian Bondy said before, variables ("int number", "char string[10]", "float your_boat", etc.) go away when then fall out of scope, like when your code leaves a function block.  So the pointer in your question ("temp") doesn't go away when you call free() -- rather, whatever your code allocated when it called malloc() goes away.  Your pointer still stays there, i.e. immediately after your example code you could say "temp = &some_other_variable" without having to say (again) "int *temp;".
If somebody ever implemented a function, that they also happened to call malloc(), that would claim memory for your program, and that did not require you to release that data, then you would be able to say
int * temp = (int*)malloc(sizeof(int));

without later saying
free(temp);

But that's not the way malloc() is implemented.

A: You don't always have to use free on a pointer, just ones declared with malloc.  You can declare a pointer that points to a memory location on the stack
int a = 3;
int* p = &a;

and this memory (along with the pointer) will also be automatically disposed of when it goes out of scope.  Using malloc allocates the same memory on the heap, so you have to handle cleanup manually.

A: Further to points made this post clarifies things further.

A: That is a very good question, while many would answer it is the difference between stack and heap allocation, the fundamental answer is the under lying system has exposed something to you that it shouldn't.
When you allocate memory, you shouldn't have to worry about giving it back.  The system should be smart enough to figure out that you have no access (point or reference) to it any more, therefore it can automatically take the memory back.
Newer languages like Java and C# have done this.
