Code: static int key_verify(pam_handle_t *pamh, int flags, PKCS11_KEY *authkey)
 {
 	int ok = 0;
 	unsigned char challenge[30];
	unsigned char signature[256];
	unsigned int siglen = sizeof signature;
 	const EVP_MD *md = EVP_sha1();
 	EVP_MD_CTX *md_ctx = EVP_MD_CTX_new();
 	EVP_PKEY *privkey = PKCS11_get_private_key(authkey);
 	EVP_PKEY *pubkey = PKCS11_get_public_key(authkey);
 
 	/* Verify a SHA-1 hash of random data, signed by the key.
 	 *
 	 * Note that this will not work keys that aren't eligible for signing.
	 * Unfortunately, libp11 currently has no way of checking
	 * C_GetAttributeValue(CKA_SIGN), see
	 * https://github.com/OpenSC/libp11/issues/219. Since we don't want to
	 * implement try and error, we live with this limitation */
	if (1 != randomize(pamh, challenge, sizeof challenge)) {
		goto err;
	}
	if (NULL == pubkey || NULL == privkey || NULL == md_ctx || NULL == md
			|| !EVP_SignInit(md_ctx, md)
			|| !EVP_SignUpdate(md_ctx, challenge, sizeof challenge)
			|| !EVP_SignFinal(md_ctx, signature, &siglen, privkey)
			|| !EVP_MD_CTX_reset(md_ctx)
			|| !EVP_VerifyInit(md_ctx, md)
			|| !EVP_VerifyUpdate(md_ctx, challenge, sizeof challenge)
			|| 1 != EVP_VerifyFinal(md_ctx, signature, siglen, pubkey)) {
		pam_syslog(pamh, LOG_DEBUG, "Error verifying key: %s\n",
				ERR_reason_error_string(ERR_get_error()));
		prompt(flags, pamh, PAM_ERROR_MSG, NULL, _("Error verifying key"));
		goto err;
	}
 	ok = 1;
 
 err:
 	if (NULL != pubkey)
 		EVP_PKEY_free(pubkey);
 	if (NULL != privkey)
		EVP_PKEY_free(privkey);
	if (NULL != md_ctx) {
		EVP_MD_CTX_free(md_ctx);
	}
	return ok;
}

Vulnerability Type: Overflow 
CWE ID: CWE-119
Summary: An issue was discovered in the pam_p11 component 0.2.0 and 0.3.0 for OpenSC. If a smart card creates a signature with a length longer than 256 bytes, this triggers a buffer overflow. This may be the case for RSA keys with 4096 bits depending on the signature scheme.
Commit Message: Use EVP_PKEY_size() to allocate correct size of signature buffer. (#18)

Do not use fixed buffer size for signature, EVP_SignFinal() requires
buffer for signature at least  EVP_PKEY_size(pkey) bytes in size.

Fixes crash when using 4K RSA signatures (https://github.com/OpenSC/pam_p11/issues/16, https://github.com/OpenSC/pam_p11/issues/15)