It seems that they are mutually exclusive, as disabling one gives me the other, and vice versa.  Two-factor auth for my ssh servers sounds really nice, so is there any way to accomplish this?
With recent Fedora and RHEL 6 releases, you can use RequiredAuthentications2 pubkey,password to require both pubkey and password authentication. Usually this is done to require pubkey and 2-factor authentication token, not the user's password.

Update: Now on RHEL / CentOS 7, and any system with a recent version of OpenSSH, you can use:

AuthenticationMethods "publickey,password" "publickey,keyboard-interactive"


It's also possible to use the Match directive to exclude IPs or Users.
