On the surface, the inadvisability of security through obscurity is directly at odds with the concept of shared secrets (i.e. "passwords"). Which is to say: if secrecy around passwords is valuable, then by extension surely it must be of some value to keep the algorithm that uses the password secret as well. Many (arguably misguided) organizations may even go as far as to say that the system is the password.

But to what degree can the secrecy of an algorithm be relied upon? Or perhaps more appropriately, in what way is secrecy surrounding an algorithm doomed to fail while secrecy surrounding passwords is not?

If, on the other hand, secrecy of an algorithm is desirable, then how important is it? To what lengths should a developer reasonably go to keep his crypto secret?

EDIT
To be clear, this isn't about creating a new, untested algorithm, but rather keeping secret the details of which algorithm you choose. For example, the technique Windows uses to hash passwords is to apply known hashing algorithms in a sequence which is not published and which appears to change between different versions of Windows.

Much of the work on passwords and keys is related to controlling where they are stored and copied.


A password is stored in the mind of a human user. It is entered on a keyboard (or equivalent) and goes through the registers of a CPU and the RAM of the computer, while it is processed. Unless some awful blunder is done, the password never reaches a permanent storage area like a hard disk.
An algorithm exists as source code somewhere, on the machine of a developer, a source versioning system, and backups. There are design documents, which have been shown to various people (e.g. those who decide whether to fund the development of the system or not), and often neglectfully deposited on an anonymous shelf of in some layer of crust on the typical desktop. More importantly, the algorithm also exists as some executable file on the deployed system itself; binary is not as readable as source code but reverse engineering works nevertheless.


Therefore we cannot reasonably consider that the algorithm is secret, or at least as secret as the password (or the key).

Really, cryptographic methods were split one century ago into the algorithm and the key precisely because of that: in a functioning system, part of the method necessarily leaks traces everywhere. Having a key means concentrating the secrecy in the other half, the part which we can keep secret.



"Security through obscurity" is an expression which uses the term obscurity, not secrecy. Cryptography is about achieving security through secrecy. That's the whole difference: a password can be secret; an algorithm is, at best, obscure. Obscurity is dispelled as soon as some smart guy thinks about bringing a metaphorical lantern. Secrecy is more like a steel safe: to break through it, you need more powerful tools.

Smart guy Auguste Kerckhoffs already wrote it more than a century ago. Despite the invention of the computer and all of today's technology, his findings still apply. It took a while for practitioners of cryptography to learn that lesson; 60 years later, Germans were still putting a great deal in the "secrecy" of the design of the Enigma machine. Note that when Germans put the 4-rotor Navy Enigma into use, Allied cryptographers were inconvenienced (routine cracking stopped for a few months) but were not totally baffled because some captured documents from the preceding year alluded to the development of the new version, with a fourth "reflector" rotor. There you have it: algorithm secrecy could not be achieved in practice.



An additional twist is that algorithm obscurity can harm security. What I explain above is that obscurity cannot be trusted for security: it might increase security, but not by much (and you cannot really know "how much"). It turns out that it can also decrease security. The problem is the following: it is very hard to make a secure cryptographic algorithm. The only known method is to publish the algorithm and wait for the collective wisdom of cryptographers around the world to gnaw at it and reach a conclusion which can be expressed as either "can be broken that way" or "apparently robust". An algorithm is declared "good" only if it resisted the onslaught of dozens or hundreds of competent cryptographers for at least three or four years.

Internet, academic procrastination and human hubris are such that, with the right communications campaign, you can get these few hundreds of cryptographers to do that hard assessing job for free -- provided that you make the algorithm public (and "attractive" in some way). If you want to maintain the algorithm obscure, then you cannot benefit from such free consulting. Instead, you have to pay. Twenty good cryptographers for, say, two years of effort: we are talking about millions of dollars, here. Nobody does that, that's way too expensive. Correspondingly, obscure algorithms are invariably much less stress-tested than public algorithms, and therefore less secure.

(Note the fine print: security is not only about not being broken, but also about having some a priori knowledge that breaches won't happen. I want to be able to sleep at night.)



Summary:


You should not keep your algorithm secret.
You do not know how much your algorithm is secret.
You cannot keep your algorithm secret.
But you can and must keep your password secret, and you can know "how much" secret it is (that's all the "entropy" business).

