Naive perhaps, but 


Why is high input impedance a good thing? 
Is high input impedance always a good thing?


It is a good thing for a voltage input, as if the input impedance is high compared to the source impedance then the voltage level will not drop too much due to the divider effect.  

For example, say we have a \$10V\$ signal with \$1k\Omega\$ impedance.   

We connect this to a \$1M\Omega\$ input, the input voltage will be \$ 10V\cdot\frac{1M\Omega}{1M\Omega+1k\Omega} = 9.99V \$.  

If we reduce the input impedance to \$10k\Omega\$, we get \$10V \cdot \frac{10k\Omega}{10k\Omega + 1k\Omega} = 9.09V\$  

Reduce it to 1k and we get \$ 10V \cdot \frac{1k\Omega}{1k\Omega + 1k\Omega} = 5V\$ 

Hopefully you get the picture - generally an input impedance of at least 10 times the source impedance is a good idea to prevent significant loading.

High input impedance is not always a good thing though, for example if you want to transfer as much power as possible then the source and load impedance should be equal. So in the above example the 1k input impedance would be the best choice.
For a current input a low input impedance (ideally zero) is desired, for example in a transimpedance (current to voltage) amplifier.
