I've been using Terminal under Mac OS X for years but somehow missed to spot this feature:  



I'm now wondering how does this actually work, and is it 100% safe? If it isn't, what technique could be used to still get the keystrokes?

"Secure Keyboard Entry" maps to the EnableSecureEventInput function whose concept is described here. Basically, applications don't access the hardware themselves; they obtain events (e.g. about key strokes) from the operating system. Some elements in the OS decides what application gets what events, depending on its access rights and GUI state (there are details depending on which application is "in the foreground").
Applications can "spy" on each other, meaning (in this case) that an application running on the machine can ask to the OS to send it a copy of all key strokes even if they are meant for another application, and/or to inject synthetic events of its own. This is a feature: it allows things like "password wallets" (which enter a password as if it was typed by the user, from the point of view of the application) or the "Keyboard Viewer" (the GUI-based keyboard which allows you to "type" characters with the mouse and also shows what keys are actually being pressed at any time). EnableSecureEventInput blocks this feature for the application which calls it. Try it ! Run Terminal.app, enable the "Keyboard Viewer", and see that enabling "Secure Keyboard Entry" prevents the Keyboard Viewer from doing its job.
All these event routing is done in some user-space process which runs as root. This relies on the process separation enforced by the kernel: normal user process cannot fiddle at will with the memory allocated for a root process. The kernel itself is unaware of the user-level concept of "event". The management of events, in particular the enforcement (or not) of EnableSecureEventInput, is made by non-kernel code.
An interesting excerpt of the page linked above is the following:

The original implementation of EnableSecureEventInput was such that when a process enabled secure input entry and had keyboard focus, keyboard events were not passed to intercept processes. However, if the secure entry process was moved to the background, the system would continue to pass keyboard events to these intercept processes, since the keyboard focus was no longer to a secure entry process.
Recently, a security hole was found that made it possible for an intercept process to capture keyboard events, even in cases where secure event input was enabled and the secure event input process was in the background. The fix for this problem is to stop passing keyboard events to any intercept process whenever any process has enabled secure event input, whether that process is in the foreground or background. This means that a process which enables secure event input and leaves secure event input enabled for the duration of the program, can affect all keyboard intercept processes, even when the secure event process has been moved to the background.

This means that the event routing system actually got it wrong in the first installment of the feature. This is now supposed to be fixed.
Even assuming that the event routing is now proper and secure, meaning that EnableSecureEventInput's semantics are really enforced, then you must understand that this is completely relative to the process separation system. Any root process can inspect and modify at will the memory of all other process, and in particular see all the events; and a root process can also hook into the kernel and inspect the actual data from the keyboard bypassing the notion of event completely. A key logger which can be installed as root will do just that, and "Secure Keyboard Entry" will be defenceless against it. See this for an opensource proof of concept.
So "Secure Keyboard Entry" is secure only against attackers who could get to run some code of their own on the machine, but could not escalate their local privileges to root level. This is a rather restrictive scenario, because local privilege escalation tends to be possible on a general basis:

Local process can see a lot of the machine, so the "security perimeter" to defend is huge in that case. Preventing intrusion from remote attackers is much easier, and yet already quite hard.

Apple tends to exhibit some lack of reactivity in the case of local privilege escalation holes.



Summary: I would find it overly optimistic to believe that "Secure Keyboard Entry" provides sufficient security against key loggers on, say, public shared computers. It is not a bad feature, but it fulfills its promises only if root and the kernel are free from malicious alterations, and that's a very big "if".
