RSA Security commonly uses keys of sizes 1024-bit, 2048-bit or even 3072-bit.  And most Symmetric algorithms only between 112-bit and 256-bit.  I do realize that the current keys are secure enough for today's hardware, but as computers get faster, should we not consider an insanely large key size like a million bits or so to protect ourselves against super computer systems that has not been invented yet?

So in other words what is the consequences of choosing a cipher key that is too large and why does everyone restrict their key sizes?

I dug out my copy of Applied Cryptography to answer this concerning symmetric crypto, 256 is plenty and probably will be for a long long time. Schneier explains;

Longer key lengths are better, but only up to a point. AES will have 128-bit, 192-bit, and 256-bit key lengths. This is far longer than needed for the foreseeable future. In fact, we cannot even imagine a world where 256-bit brute force searches are possible. It requires some fundamental breakthroughs in physics and our understanding of the universe.
One of the consequences of the second law of thermodynamics is that a certain amount of energy is necessary to represent information. To record a single bit by changing the state of a system requires an amount of energy no less than kT, where T is the absolute temperature of the system and k is the Boltzman constant. (Stick with me; the physics lesson is almost over.)
Given that k = 1.38 × 10−16 erg/K, and that the ambient temperature of the universe is 3.2 Kelvin, an ideal computer running at 3.2 K would consume 4.4 × 10−16 ergs every time it set or cleared a bit. To run a computer any colder than the cosmic background radiation would require extra energy to run a heat pump.
Now, the annual energy output of our sun is about 1.21 × 1041 ergs. This is enough to power about 2.7 × 1056 single bit changes on our ideal computer; enough state changes to put a 187-bit counter through all its values. If we built a Dyson sphere around the sun and captured all its energy for 32 years, without any loss, we could power a computer to count up to 2192. Of course, it wouldn't have the energy left over to perform any useful calculations with this counter.
But that's just one star, and a measly one at that. A typical supernova releases something like 1051 ergs. (About a hundred times as much energy would be released in the form of neutrinos, but let them go for now.) If all of this energy could be channeled into a single orgy of computation, a 219-bit counter could be cycled through all of its states.
These numbers have nothing to do with the technology of the devices; they are the maximums that thermodynamics will allow. And they strongly imply that brute-force attacks against 256-bit keys will be infeasible until computers are built from something other than matter and occupy something other than space.

The boldness is my own addition.
Remark: Note that this example assumes that there is a 'perfect' encryption algorithm. If you can exploit weaknesses in the algorithm, the key space might shrink and you'd end up with effectively less bits of your key.
It also assumes that the key generation is perfect - yielding 1 bit of entropy per bit of key. This is often difficult to achieve in a computational setting. An imperfect generation mechanism might yield 170 bits of entropy for a 256 bit key. In this case, if the key generation mechanism is known, the size of the brute-force space is reduced to 170 bits.
Assuming quantum computers are feasible, however, any RSA key will be broken using Shor's algorithm. (See https://security.stackexchange.com/a/37638/18064)
