I know that one shouldn't rely on "obscurity" for their security. For example, choosing a non-standard port is not really security, but it also doesn't usually hurt to do so (and may help mitigate some of the most trivial attacks).

Hashing and encryption relies on strong randomization and secret keys. RSA, for instance, relies on the secrecy of d, and by extension, p, q, and ϕ(N). Since those have to be kept secret, isn't all encryption (and hashing, if you know the randomization vector) security through obscurity? If not, what is the difference between obscuring the secret sauce and just keeping the secret stuff secret? The reason we call (proper) encryption secure is because the math is irrefutable: it is computationally hard to, for instance, factor N to figure out p and q (as far as we know). But that's only true because p and q aren't known. They're basically obscured.

I've read The valid role of obscurity and At what point does something count as 'security through obscurity'?, and my question is different because I'm not asking about how obscurity is valid or when in the spectrum a scheme becomes obscure, but rather, I'm asking if hiding all our secret stuff isn't itself obscurity, even though we define our security to be achieved through such mechanisms. To clarify what I mean, the latter question's answers (excellent, by the way) seem to stop at "...they still need to crack the password" -- meaning that the password is still obscured from the attacker.

See this answer.

The main point is that we make a sharp distinction between obscurity and secrecy; if we must narrow the difference down to a single property, then that must be measurability. Is secret that which is not known to outsiders, and we know how much it is unknown to these outsiders. For instance, a 128-bit symmetric key is a sequence of 128 bits, such that all 2128 possible sequences would stand an equal probability of being used, so the attacker trying to guess such a key needs to try out, on average, at least 2127 of them before hitting the right one. That's quantitative. We can do math, add figures, and compute attack cost.

The same would apply to a RSA private key. The maths are more complex because the most effective known methods rely on integer factorization and the involved algorithms are not as easy to quantify as brute force on a symmetric key (there are a lot of details on RAM usage and parallelism or lack thereof). But that's still secrecy.

By contrast, an obscure algorithm is "secret" only as long as the attacker does not work out the algorithm details, and that depends on a lot of factors: accessibility to hardware implementing the algorithm, skills at reverse-engineering, and smartness. We do not have a useful way to measure how smart someone can be. So a secret algorithm cannot be "secret". We have another term for that, and that's "obscure".

We want to do security through secrecy because security is risk management: we accept the overhead of using a security system because we can measure how much it costs us to use it, and how much it reduces the risk of successful attacks, and we can then balance the costs to take an informed decision. This may work only because we can put numbers on risks of successful attacks, and this can be done only with secrecy, not with obscurity.
