Detach encrypt and keygen hooks from RSA_METHOD.
Nothing is using them. For encrypt, there's generally no need to swap
out public key operations. keygen seems especially pointless as one
could just as easily call the other function directly.
The one behavior change is RSA_encrypt now gracefully detects if called
on an empty RSA, to match the other un-RSA_METHOD-ed functions which had
similar treatments. (Conscrypt was filling in the encrypt function
purely to provide a non-crashing no-op function. They leave the public
bits blank and pass their custom keys through sufficiently many layers
of Java crypto goo that it's not obvious whether this is reachable.)
We still can't take the function pointers out, but once
https://github.com/google/conscrypt/commit/96bbe03dfd2737f0c1461db59966ff41502a91e4
trickles back into everything, we can finally prune RSA_METHOD.
Bump BORINGSSL_API_VERSION as a convenience so I can land the
corresponding removal in Conscrypt immediately.
Change-Id: Ia2ef4780a5dfcb869b224e1ff632daab8d378b2e
Reviewed-on: https://boringssl-review.googlesource.com/15864
Reviewed-by: Adam Langley <agl@google.com>
Commit-Queue: Adam Langley <agl@google.com>
CQ-Verified: CQ bot account: commit-bot@chromium.org <commit-bot@chromium.org>
diff --git a/include/openssl/rsa.h b/include/openssl/rsa.h
index c5510c8..b697a7d 100644
--- a/include/openssl/rsa.h
+++ b/include/openssl/rsa.h
@@ -583,13 +583,13 @@
int (*verify)(int dtype, const uint8_t *m, unsigned int m_length,
const uint8_t *sigbuf, unsigned int siglen, const RSA *rsa);
-
- /* These functions mirror the |RSA_*| functions of the same name. */
+ /* Ignored. Set this to NULL. */
int (*encrypt)(RSA *rsa, size_t *out_len, uint8_t *out, size_t max_out,
const uint8_t *in, size_t in_len, int padding);
+
+ /* These functions mirror the |RSA_*| functions of the same name. */
int (*sign_raw)(RSA *rsa, size_t *out_len, uint8_t *out, size_t max_out,
const uint8_t *in, size_t in_len, int padding);
-
int (*decrypt)(RSA *rsa, size_t *out_len, uint8_t *out, size_t max_out,
const uint8_t *in, size_t in_len, int padding);
/* Ignored. Set this to NULL. */
@@ -621,6 +621,7 @@
int flags;
+ /* Ignored. Set this to NULL. */
int (*keygen)(RSA *rsa, int bits, BIGNUM *e, BN_GENCB *cb);
/* Ignored. Set this to NULL. */