CMS KEMRecipientInfo (RFC 9629): three unchecked exceptions on malformed fields; kekLength not checked against wrap

[Arpan0995]

Summary

On the recipient side of the RFC 9629 KEMRecipientInfo path, three sender-controlled fields produce an unchecked exception out of an API declared throws CMSException. Two of them fire at parse time in every CMSEnvelopedData and CMSAuthEnvelopedData constructor and streaming parser, before any key is involved; the third fires at unwrap. In each case the malformed input is rejected either way, so nothing decrypts that should not, but the exception type escapes the documented contract. This is the recipient-side counterpart of #2398, which was on the generator side: here the fields arrive in the message, and the methods they escape from are declared throws CMSException. Separately, the recipient never checks the wire kekLength against the wrap algorithm, which RFC 9629 states as a MUST. All four are in the RFC 9629 recipient path and the fixes touch KEMRecipientInfo and JceKEMRecipient only, so I have kept them together.

Environment

• bcprov / bcutil / bcpkix 1.86.0.20698 (current 1.86 beta), main at commit 402aed6
• JDK 27

Steps to reproduce

Build one valid ML-KEM-768 EnvelopedData, then rewrite individual KEMRecipientInfo fields at the ASN.1 level:

import java.io.ByteArrayInputStream;
import java.math.BigInteger;
import java.security.KeyPair;
import java.security.KeyPairGenerator;
import java.security.Security;
import java.util.ArrayList;
import java.util.List;
import org.bouncycastle.asn1.*;
import org.bouncycastle.asn1.cms.CMSObjectIdentifiers;
import org.bouncycastle.asn1.cms.ContentInfo;
import org.bouncycastle.asn1.cms.EnvelopedData;
import org.bouncycastle.asn1.cms.OtherRecipientInfo;
import org.bouncycastle.asn1.cms.RecipientInfo;
import org.bouncycastle.asn1.pkcs.PKCSObjectIdentifiers;
import org.bouncycastle.asn1.x509.AlgorithmIdentifier;
import org.bouncycastle.cms.*;
import org.bouncycastle.cms.jcajce.JceCMSContentEncryptorBuilder;
import org.bouncycastle.cms.jcajce.JceKEMEnvelopedRecipient;
import org.bouncycastle.cms.jcajce.JceKEMRecipientInfoGenerator;
import org.bouncycastle.jce.provider.BouncyCastleProvider;

public class CmsKemRecipientInfoRepro {
    public static void main(String[] args) throws Exception {
        Security.addProvider(new BouncyCastleProvider());
        KeyPair kp = KeyPairGenerator.getInstance("ML-KEM-768", "BC").generateKeyPair();

        // a valid ML-KEM-768 EnvelopedData with one KEMRecipientInfo (AES-256-KW wrap)
        CMSEnvelopedDataGenerator gen = new CMSEnvelopedDataGenerator();
        gen.addRecipientInfoGenerator(new JceKEMRecipientInfoGenerator(new byte[]{1, 2, 3, 4}, kp.getPublic(), CMSAlgorithm.AES256_WRAP).setProvider("BC"));
        byte[] good = gen.generate(new CMSProcessableByteArray("hello".getBytes()),
            new JceCMSContentEncryptorBuilder(CMSAlgorithm.AES256_CBC).setProvider("BC").build()).getEncoded();

        // pull the KEMRecipientInfo SEQUENCE out so individual fields can be rewritten
        EnvelopedData env = EnvelopedData.getInstance(ContentInfo.getInstance(good).getContent());
        RecipientInfo ri = RecipientInfo.getInstance(env.getRecipientInfos().getObjectAt(0));
        ASN1Sequence kem = ASN1Sequence.getInstance(OtherRecipientInfo.getInstance(ri.getInfo()).getValue());
        List<ASN1Encodable> el = new ArrayList<ASN1Encodable>();
        for (int i = 0; i != kem.size(); i++) el.add(kem.getObjectAt(i));
        // elements: [0]version [1]rid [2]kem [3]kemct [4]kdf [5]kekLength [6]wrap [7]encryptedKey

        // D1: wrap = an OID CMSUtils.getKekSize does not know (BC accepts it elsewhere); parses fine, then getContent()
        List<ASN1Encodable> d1 = new ArrayList<ASN1Encodable>(el);
        d1.set(6, new AlgorithmIdentifier(PKCSObjectIdentifiers.id_alg_CMS3DESwrap));
        try {
            RecipientInformation r = (RecipientInformation) new CMSEnvelopedData(rebuild(env, d1)).getRecipientInfos().getRecipients().iterator().next();
            r.getContent(new JceKEMEnvelopedRecipient(kp.getPrivate()).setProvider("BC"));
        } catch (Throwable t) { report("D1 unsupported wrap OID, RecipientInformation.getContent()", t); }

        // D2: kekLength = 2^40, encoded in more than four content bytes; intValueExact rejects it before the 1..65535 range check runs
        List<ASN1Encodable> d2 = new ArrayList<ASN1Encodable>(el);
        d2.set(5, new ASN1Integer(BigInteger.ONE.shiftLeft(40)));
        try { new CMSEnvelopedData(rebuild(env, d2)); } catch (Throwable t) { report("D2 kekLength = 2^40, new CMSEnvelopedData(byte[])", t); }
        try { new CMSEnvelopedDataParser(new ByteArrayInputStream(rebuild(env, d2))); } catch (Throwable t) { report("D2 kekLength = 2^40, CMSEnvelopedDataParser", t); }

        // D3: 8 elements with a [0]-tagged ukm at index 6 and no encryptedKey; size check passes, index 8 is read
        List<ASN1Encodable> d3 = new ArrayList<ASN1Encodable>(el.subList(0, 6));
        d3.add(new DERTaggedObject(true, 0, new DEROctetString(new byte[]{9, 9})));
        d3.add(el.get(6));
        try { new CMSEnvelopedData(rebuild(env, d3)); } catch (Throwable t) { report("D3 8 elements with [0] ukm, new CMSEnvelopedData(byte[])", t); }
    }

    static byte[] rebuild(EnvelopedData env, List<ASN1Encodable> el) throws Exception {
        ASN1EncodableVector v = new ASN1EncodableVector();
        for (ASN1Encodable e : el) v.add(e);
        RecipientInfo ri = new RecipientInfo(new OtherRecipientInfo(CMSObjectIdentifiers.id_ori_kem, new DERSequence(v)));
        EnvelopedData n = new EnvelopedData(env.getOriginatorInfo(), new DERSet(ri), env.getEncryptedContentInfo(), env.getUnprotectedAttrs());
        return new ContentInfo(CMSObjectIdentifiers.envelopedData, n).getEncoded();
    }

    static void report(String label, Throwable t) {
        System.out.println(label + "\n    -> " + t.getClass().getName() + ": " + t.getMessage());
    }
}

Actual behaviour

D1 unsupported wrap OID, RecipientInformation.getContent()
    -> java.lang.IllegalArgumentException: unknown wrap algorithm
D2 kekLength = 2^40, new CMSEnvelopedData(byte[])
    -> java.lang.ArithmeticException: ASN.1 Integer out of int range
D2 kekLength = 2^40, CMSEnvelopedDataParser
    -> java.lang.ArithmeticException: ASN.1 Integer out of int range
D3 8 elements with [0] ukm, new CMSEnvelopedData(byte[])
    -> java.lang.ArrayIndexOutOfBoundsException: Index 8 out of bounds for length 8

RecipientInformation.getContent, new CMSEnvelopedData(byte[]) and CMSEnvelopedDataParser are all declared throws CMSException. The D2 and D3 exceptions also escape new CMSAuthEnvelopedData(byte[]) and CMSAuthEnvelopedDataParser, which share the same KEMRecipientInfo decoding.

Expected behaviour

A CMSException in each case. new CMSEnvelopedData(byte[]) already does this for the neighbouring inputs: a kekLength of 0, -1 or 65536 gives CMSException: Malformed content. from it, and a wrap OID that is supported but does not match the ciphertext (AES-256-KWP on an AES-KW encrypted key) gives CMSException: exception unwrapping key ... checksum failed from getContent. Through new CMSAuthEnvelopedData(byte[]) and both streaming parsers those same in-range values already escape as a raw IllegalArgumentException; that is a general gap not specific to KEM, which I will file separately.

Root cause

D1, unsupported wrap OID at unwrap. JceKEMRecipient.extractSecretKey calls helper.createKEMUnwrapper(...) at JceKEMRecipient.java:189, outside the try at lines 201-215, which catches only OperatorException. The JceCMSKEMKeyUnwrapper constructor (JceCMSKEMKeyUnwrapper.java:42) immediately calls CMSUtils.getKekSize (cms/jcajce/CMSUtils.java:266-285), which knows the six AES-KW and AES-KWP OIDs and otherwise throws IllegalArgumentException("unknown wrap algorithm") at line 283 (the method carries a TODO: add table). RFC 9629 leaves the wrap algorithm identifier open-ended, so an unsupported one is an ordinary condition to report through the contract.

D2, oversized kekLength at parse. The KEMRecipientInfo constructor (KEMRecipientInfo.java:106 and :110) calls kekLength.intValueExact() in order to apply the 1..65535 range check, but ASN1Integer.intValueExact (ASN1Integer.java:243-252) throws ArithmeticException for any integer with more than four content bytes before the comparison happens, so the range check that exists to reject such values never runs. new CMSEnvelopedData translates ClassCastException and IllegalArgumentException from recipient parsing to CMSException (CMSEnvelopedData.java:133-140), but not ArithmeticException; new CMSAuthEnvelopedData translates none of them, since its equivalent catch (CMSAuthEnvelopedData.java:79-90) covers only AuthEnvelopedData.getInstance and the recipient store is built outside it; and the CMSEnvelopedDataParser and CMSAuthEnvelopedDataParser constructors have no try at all. RFC 9629 section 3 says of the bound: "The upper bound on the integer value is provided to make it clear to implementers that support for very large integer values is not needed." Rejection is intended; it is only the exception type that is wrong.

D3, ukm present with encryptedKey absent, at parse. KEMRecipientInfo.java:90 accepts a sequence of size 8 or 9 unconditionally. Lines 116-125 then consume a [0]-tagged element 6 as ukm and read wrap at index 7 and encryptedKey at index 8 regardless of the actual size, so an 8-element sequence whose element 6 is tagged reaches ASN1Sequence.getObjectAt (ASN1Sequence.java:292-295, a bare elements[index]) with index 8. The mirror image is lenient the other way: a 9-element sequence with no tagged element 6 parses, silently ignores the trailing element, and decrypts. The size check is not tied to whether ukm is present.

kekLength not checked against wrap (conformance). JceCMSKEMKeyUnwrapper.java:42 derives kekLength from the wrap OID and line 98 builds CMSORIforKEMOtherInfo from that derived value; the wire kekLength is never compared to it. KEMRecipientInfo in fact exposes no accessor for it: the value is decoded, range-checked and discarded (the accessors cover the recipient identifier, kem, kemct, kdf, wrap, ukm and encryptedKey), which is also why nothing downstream can check it. A kekLength of 16 with AES-256-KW is accepted and decrypts. RFC 9629 section 3: "Implementations MUST confirm that the value provided is consistent with the key-encryption algorithm identified in the wrap field below." There is no security consequence, since the KDF binds the value BC actually uses and a sender with a mismatching value fails to unwrap, but the MUST is unmet. JceKEMRecipient.java:187 already decodes the KEMRecipientInfo at the point where the comparison belongs.

Impact

Robustness and contract. D2 and D3 are reachable by anyone who can hand the application a CMS message to parse; no key is needed and the exception fires before any recipient is selected, so a single malformed RecipientInfo in a multi-recipient message ends the parse with an exception the caller was not told to expect. D1 needs the caller to attempt decryption. Everything fails closed. The project already treats this as a defect in the same constructor: the comment at CMSEnvelopedData.java:112-115 explains that an escaping NullPointerException "would escape this ctor's declared throws CMSException" and reports it as CMSException for exactly that reason.

Suggested direction

• D1: catch IllegalArgumentException around the createKEMUnwrapper call at JceKEMRecipient.java:189 and rethrow it as CMSException naming the wrap OID; getKekSize can keep throwing IllegalArgumentException, and could name the OID as the CMS RFC 9629 KEM: getEncapsulationLength throws NullPointerException for a KEM absent from its table (e.g. a BIKE recipient key) #2398 fix did for the KEM. Moving the call inside the existing try is not enough on its own, since that try catches only OperatorException.
• D2: test the kekLength range on the BigInteger (or on the encoded length) before calling intValueExact, so an oversized value takes the IllegalArgumentException path. CMSAuthEnvelopedData and the two streaming parsers could also be given the same ClassCastException / IllegalArgumentException translation new CMSEnvelopedData has, though that is the general gap mentioned above rather than a KEM item.
• D3: tie the size check to the ukm tag: a 9-element sequence requires a tagged element 6, an 8-element sequence requires an untagged one. No conformant encoder emits an extra trailing element in the SEQUENCE, so the tightening cannot reject a valid message.
• kekLength: add a getKekLength() accessor to KEMRecipientInfo, then compare it with unwrapper.getKekLength() (already public on JceCMSKEMKeyUnwrapper) in JceKEMRecipient.extractSecretKey and reject a mismatch with CMSException.

The program above is complete and self-contained; it needs bcprov, bcutil and bcpkix on the classpath.

[dghgit]

Thanks for the report. This is now fixed and up on https://downloads.bouncycastle.org/betas