Add support for verifying S/MIME messages by nitneuqr · Pull Request #12267 · pyca/cryptography

nitneuqr · 2025-01-10T14:32:30Z

As promised in #11555, I'm opening this PR with an initial implementation of S/MIME verification, in order to better discuss the API design, and to start the reviews while I finish some other features.

Namely, the new pkcs7_verify functions do not handle the certificate verification feature as of now. It as similar to a openssl smime -verify with the -noverify flag, to verify the signature but not the certificates (similar to what #12116 needs). Can you point me towards some existing code verifying X.509 certificates, if some exists?

Also, I have one question about the certificate parameter in the functions: should we verify against one certificates? Multiple ones? All the ones that are stored in the signature (if any)?

My essential thoughts for testing were to do the round-trip: signature using the PKCS7SignatureBuilder and verifying using the pkcs_decrypt functions. For now, I've not replaced the test_support.pkcs7_verify function, but I'm planning to do so as soon as the certificate verification feature is developed.

I'm still new to rust, so please let me know if you see some issues in variable lifetime, or some unnecessary copying between Python & Rust.

cc @alex

alex · 2025-01-10T14:34:41Z

The existing apis for cert verification are in cryptography.x509.verification

…

On Fri, Jan 10, 2025, 9:32 AM Quentin Retourne ***@***.***> wrote: As promised in #11555 <#11555>, I'm opening this PR with an initial implementation of S/MIME verification, in order to better discuss the API design, and to start the reviews while I finish some other features. Namely, the new pkcs7_verify functions do not handle the certificate verification feature as of now. It as similar to a openssl smime -verify with the -noverify flag, to verify the signature but not the certificates (similar to what #12116 <#12116> needs). Can you point me towards some existing code verifying X.509 certificates, if some exists? Also, I have one question about the certificate parameter in the functions: should we verify against one certificates? Multiple ones? All the ones that are stored in the signature (if any)? My essential thoughts for testing were to do the round-trip: signature using the PKCS7SignatureBuilder and verifying using the pkcs_decrypt functions. For now, I've not replaced the test_support.pkcs7_verify function, but I'm planning to do so as soon as the certificate verification feature is developed. I'm still new to rust, so please let me know if you see some issues in variable lifetime, or some unnecessary copying between Python & Rust. cc @alex <https://github.com/alex> ------------------------------ You can view, comment on, or merge this pull request online at: #12267 Commit Summary - 794db04 <794db04> PKCS7 signing (first version) File Changes (9 files <https://github.com/pyca/cryptography/pull/12267/files>) - *M* CHANGELOG.rst <https://github.com/pyca/cryptography/pull/12267/files#diff-2c623f3c6a917be56c59d43279244996836262cb1e12d9d0786c9c49eef6b43c> (4) - *M* docs/development/test-vectors.rst <https://github.com/pyca/cryptography/pull/12267/files#diff-abc63e746ffc66f83cffdf369028671e53de77b100eb528f517f16f41201f3f7> (3) - *M* docs/hazmat/primitives/asymmetric/serialization.rst <https://github.com/pyca/cryptography/pull/12267/files#diff-7c2790036c595946b546ceec3cb5dc5125b4f6223761a6684f577e70fb49d458> (148) - *M* src/cryptography/hazmat/bindings/_rust/pkcs7.pyi <https://github.com/pyca/cryptography/pull/12267/files#diff-836c9b03108b81b3659a23a89d2c5fdddab94ca204686c4e71afcb7e433a7f87> (28) - *M* src/cryptography/hazmat/primitives/serialization/pkcs7.py <https://github.com/pyca/cryptography/pull/12267/files#diff-0075be2b09cefb4b36a4646255d3e1bea05c99b6e3ffd141a08aa528de9741c3> (34) - *M* src/rust/src/pkcs7.rs <https://github.com/pyca/cryptography/pull/12267/files#diff-8ca3ed0edb153cd77f373e5e5856e6f1fc559cddcc5ebf91cc690a7aed7f0a3a> (167) - *M* src/rust/src/types.rs <https://github.com/pyca/cryptography/pull/12267/files#diff-4eb6bcd40dbc27bf39e7b1c1e694d48c28fec6579d4ab1cb0a7b3ad65f9d2228> (5) - *M* tests/hazmat/primitives/test_pkcs7.py <https://github.com/pyca/cryptography/pull/12267/files#diff-ee3cc87634fba3d98c17ee2dbcb575f63f33603a5ab7ac9fa7bcc3b17bced6a3> (169) - *A* vectors/cryptography_vectors/pkcs7/signed-opaque.msg <https://github.com/pyca/cryptography/pull/12267/files#diff-566d9755c6ec5055a0a4ca69a7a0b94f91d23d42491be4d6a591e88fad57f0f4> (21) Patch Links: - https://github.com/pyca/cryptography/pull/12267.patch - https://github.com/pyca/cryptography/pull/12267.diff — Reply to this email directly, view it on GitHub <#12267>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAAGBF2ULFD3GXMBFEJ3432J7KZJAVCNFSM6AAAAABU6QAS7KVHI2DSMVQWIX3LMV43ASLTON2WKOZSG44DAMZSG42DQNY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

nitneuqr · 2025-01-10T14:34:52Z

+            raise ValueError(
+                "Malformed multipart/signed message: must be multipart"
+            )
+        if not isinstance(payload[0], email.message.Message):


Here, I'm not managing to create any test case (specific MIME message) that would raise this error. However, according to mypy, the case is still possible. Should I go with an assert in order to please mypy or have you got any idea on how the parts could be some text and not an email.message.Message?

nitneuqr · 2025-01-11T09:53:37Z

@alex, for now I'm trying the following code:

def _verify_pkcs7_certificates(certificates: list[x509.Certificate]) -> None:
    builder = PolicyBuilder().store(Store(certificates))
    verifier = builder.build_client_verifier()
    verifier.verify(certificates[0], certificates[1:])

However, I'm getting the following error:

E       cryptography.hazmat.bindings._rust.x509.VerificationError: validation failed: basicConstraints.cA must not be asserted in an EE certificate (encountered processing <Certificate(subject=<Name(C=US,CN=cryptography CA)>, ...)>)

To be honest, I'm not clear as to what we are supposed to do in certificate verification. I've not found anything specific in the RFC, but I'll keep looking. If you can guide me through this, I'd love some help 🛩️

alex · 2025-01-13T00:26:05Z

Basically certificates used as end entities (i.e., the cert used to sign a PKCS#7/SMIME message) should not hvae ca=true in their basic constraints extension. I can't remember which spec that comes from, but its a general best practice in any event /cc @woodruffw

woodruffw · 2025-01-13T00:30:10Z

Yep, that comes from CABF -- we have a test and cite for it here: https://x509-limbo.com/testcases/webpki/#webpkica-as-leaf.

(RFC 5280 doesn't impose the requirement that the EE not be a CA, but firmly agreed about it being best practice.)

nitneuqr · 2025-01-13T07:22:43Z

Basically certificates used as end entities (i.e., the cert used to sign a PKCS#7/SMIME message) should not hvae ca=true in their basic constraints extension. I can't remember which spec that comes from, but its a general best practice in any event /cc @woodruffw

OK, that raises two things for me:

That means we will have to create a specific certificate for verifying (and signing ?) PKCS#7 files, either statically or dynamically. I'll look into it.
This raises the question of test_support.pkcs7_verify function. It does not check this, as we can see in test_support.rs:93, by just passing an empty stack of certificates.

let certs = openssl::stack::Stack::new()?;

p7.verify(
    &certs,
    &store,
    msg.as_ref().map(|m| m.as_bytes()),
    None,
    flags,
)?;

nitneuqr · 2025-01-15T15:30:34Z

@alex this should be ready for review, the tests are passing. A few considerations to keep in mind:

What should the priority be for user-input parameters vs. what's found in the PKCS7 structure (content, certificate)?
Handling PKCS#7 message with multiple signers: trying to verify the first signer? All of them?
Missing the documentation for the new pkcs#7 test vectors; still WIP.

Keep in mind that for now, I've used a certificate chain from endesive to make the tests pass, but I'll work on creating new cryptography certificates in the following days.

Some more things about the certs:

the certificate used for testing PKCS#7 signing cannot be verified properly (due to some EE error above)
Verifying the signed messages, when using openssl, only works when some other certificates are passed in the -CAfile parameter. Otherwise we are getting a unable to get local issuer certificate error. This is not necessary (for some reason that I do not understand) in the new functions.

CC @reaperhulk @facutuesca if you miraculously have time for a review 😄

nitneuqr · 2025-01-15T15:36:32Z

Stumbled on #12104 and realized we might have the same issue with the certificate verification code!

alex · 2025-01-20T15:15:37Z

Sorry I've been slow here, it's on my TODO list for this week.

…

On Mon, Jan 20, 2025 at 10:13 AM Quentin Retourne ***@***.***> wrote: @nitneuqr <https://github.com/nitneuqr> requested your review on: #12267 <#12267> Add support for verifying S/MIME messages. — Reply to this email directly, view it on GitHub <#12267 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAAGBBDZHEQAET2H44MH7L2LUHAXAVCNFSM6AAAAABU6QAS7KVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJWGAYDANBRG4ZDKMY> . You are receiving this because your review was requested.Message ID: ***@***.***>

-- All that is necessary for evil to succeed is for good people to do nothing.

nitneuqr · 2025-01-21T09:11:01Z

No worries! Just checking in in case you forgot about it 😊

prauscher · 2025-01-24T12:40:44Z

Stumbled on #12104 and realized we might have the same issue with the certificate verification code!

Not exactly sure, but first huge thanks for the work: After reading your PR, your Usecase seems to be "I want to verify a received SMIME-Message" (along what I paraphased in #12104 as verify_message-method of a Verifier to be obtained using build_smime_verifier). My usecase is more "I want to verify a Certificate to use it for sending a SMIME-Encrypted message" (described as verify_certificate).

However, my Usecase should be a subtask of your Usecase: To verify a received SMIME-Message you probably need to verify the used Certificate. Looks like you are doing this in _verify_pkcs7_certificate, which is using build_client_verifier - however from my knowledge, this requires EKU_CLIENT_AUTH_OID to be set, but SMIME-Certificates can be fine without client auth (and require EKU_EMAIL_PROTECTION_OID instead.

So in my (really basic) knowledge, your current code would give invalid validation results for Certificates which are only used for SMIME-Signing (and not client authentication). However, once you include a verifier for it, please make it publicly accessible to use it without a signed Message :)

alex · 2025-01-31T21:23:54Z

Looking at your questions:

What should the priority be for user-input parameters vs. what's found in the PKCS7 structure (content, certificate)?

What if we start with the most restrictive implementation: it is an error for both to be present?

Handling PKCS#7 message with multiple signers: trying to verify the first signer? All of them?

I think for an MVP we should start with an "any" policy: if any of the signers matches what we want, then it's ok. (In the future maybe there's a use case for an "all" policy? I'm not sure)

alex · 2025-01-31T21:29:42Z

+                let certificates = pyo3::types::PyList::empty(py);
+                certificates.append(certificate)?;


We probably need to add an argument for people to provide a trust store

nitneuqr · 2025-02-02T19:27:27Z

Waiting for #12360 before going any further on this :)

lgcCerti · 2025-02-06T12:49:16Z

Hello, I opened #12406 wondering about this implementation.
Would a PKCS7 verification be compatible with CMS? Such as the CMS_verify used in openssl-cms ?

According to OpenSSL they are very similar, but I couldn't pinpoint the edge cases

alex · 2025-02-14T17:14:43Z

FYI, #12360 is merged now.

nitneuqr · 2025-02-15T10:44:20Z

FYI, #12360 is merged now.

Will try to work on this the following weeks. I just read https://github.com/nitneuqr/cryptography/blob/pkcs7-verify/docs/x509/verification.rst#L284; as far as I understand, it seems we need to create a dedicated S/MIME extension policy and verify based on this policy?

Do you know where you can find information on the required information for S/MIME certificates?

Update: just re-read all the comments and it seems at least part of the info is there 😄

nitneuqr · 2025-02-15T11:09:40Z

As far as I understand it now, it seems that we'll need first to create a S/MIME extension policy (probably in another PR), test it with some certificates, and use the same certificates to test S/MIME verification. Does that make sense @alex?

Update: opened #12465 for that matter.

alex · 2025-02-15T15:02:24Z

That sounds like a good plan.

…

On Sat, Feb 15, 2025 at 5:10 AM Quentin Retourne ***@***.***> wrote: As far as I understand it now, it seems that we'll need first to create a S/MIME extension policy (probably in another PR), test it with some certificates, and use the same certificates to test S/MIME verification. Does that make sense @alex <https://github.com/alex>? — Reply to this email directly, view it on GitHub <#12267 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAAGBH4KMGYVMIYZAS5U332P4OAVAVCNFSM6AAAAABU6QAS7KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRQHA3TGNZUGI> . You are receiving this because you were mentioned.Message ID: ***@***.***> [image: nitneuqr]*nitneuqr* left a comment (pyca/cryptography#12267) <#12267 (comment)> As far as I understand it now, it seems that we'll need first to create a S/MIME extension policy (probably in another PR), test it with some certificates, and use the same certificates to test S/MIME verification. Does that make sense @alex <https://github.com/alex>? — Reply to this email directly, view it on GitHub <#12267 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAAGBH4KMGYVMIYZAS5U332P4OAVAVCNFSM6AAAAABU6QAS7KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRQHA3TGNZUGI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- All that is necessary for evil to succeed is for good people to do nothing.

nitneuqr · 2025-06-15T17:06:34Z

@alex this should be ready again for review as soon as #12465 is integrated :)

added tests accordingly adapted the pkcs7 certificate adapted EE policy do not know if a CA policy is needed! added SAN checking

handling PEM, DER, SMIME formats added tests & documentation accordingly doing assertions for now, to please mypy added more test coverage updated tests to avoid unsupported algorithm first failing code for certificate verification handling mixed types with Cow feat: functions now have optional keyword arguments certificate is now optional feat: handling RSA case feat: No signature parameter adapted tests accordingly fix: adapted docmentation fix: passed into Cow again coverage: added one test case one more test case changing the error message for clarity two more test cases handled loading the load_der func for all backends removed options for now first draft of smime extension policy added back changelog integrated built-in verifier using existing vectors :) minor doc modification removed old vectors

Tarnum-tst · 2026-01-09T07:34:07Z

+        )
+
+        # Verification
+        pkcs7.pkcs7_verify_smime(signed)


This call produces VerificationError in signed-opaque.msg due to an expired certificate.

TaaviE · 2026-02-13T10:28:23Z

I've spent a bit of time implementing the same on top of pyca/cryptography crates, there are three other PRs this PR might want to consider:

For S/MIME support beyond S/MIME version 2 (from the 90s), structures from Update extremely outdated structures to tolerate relevant versions of CMS and S/MIME #14286
If not incorporated, it would be reasonable to check the CMS SignedData version and throw an Exception, because the implementation will not handle it correctly.
For S/MIME certificate validation, the respective S/MIME BR policies: Add CABF S/MIME certificate policies #14190
For correct RevocationInfoChoice OCSPResponse handling: Better align structures with the OCSP RFC #14304

There's also one thing that seems actually dangerous in this implementation, the eContent handling. If it exists in SingedData and content was passed as arg, the function should probably throw. Right now (if I'm not reading it wrong) one can pass content into verify_der and the signature is checked over eContent instead. Which can result in the wrong content being considered signed.

Thing that also seems unhandled is verifying the signer address. This is difficult to do correctly and probably shouldn't be left to end-users calling the API. It should also handle UTF-8 SANs and From-s. All of this is described more in-depth in the relevant RFCs.

alex · 2026-06-09T21:28:08Z

Hi all,

First, my sincerest apologies that we didn't make progress on this for so long. PKCS#7 is not an area of deep expertise for either myself or @reaperhulk, and there was a lot to chew through, and we didn't want to get this one wrong and be stuck with some bad APIs. He and I have just sat down and talked through this, and so this is a write up of what we discussed. And also I apologize since I know we've given a lot of different API guidance over the years, hopefully we can get to a solid decision we all feel good about.

First, at a high level: Our goal is to provide something that is, a) useful to people (solves real problems), b) long term maintainable by us (accounting for our lack of PKCS#7 specific expertise), c) is consistent with our overall principles (including our goals around security).

Our next move was to try to map out "what is PKCS#7 verification", and we think, if you break it down to its roots, it's: 1) a signature, 2) signed data, 3) the public key which verifies the signature, 4) trust/identity for that key. And each of these can come from different places:

Signature: this is always internal to the PKCS#7 structure
Signed Data: this comes from one of the PKCS#7 structure, the S/MIME bundle, or provided external
Public Key: this comes either from the PKCS#7 structure's certificates, or is provided externally
Trust/Identity: this is either inherent (because the user already has a specific key they trust), or comes from X.509 path building from the certificates in the PKCS#7 structure to a (set of) root(s) the user inherently trust

Combinatorally that's 12 different operations we can be performing (and that's without multiplying by 3x for the formats: SMIME, DER, PEM) -- though not all combinations make sense.

Recognizing this complexity, we decided what we needed to do is simplify the API design space in order to make this more manageable. And here's what we came up with:

We think at the base of all of this, there's a def verify_pkcs7_directly_signed_by(key: PublicKey, pkcs7_der: bytes, contents: bytes | None) -> bytes. This function is responsible for extracting the signature and (optionally) contents from the PKCS#7 structure, and then doing the signature verification. It returns the contents. Once you have this primitive, you can build everything else on top of it.

The other major primitive you want is: def verify_pkcs7_signature_with_trust(verifier: ClientVerifier, pkcs7_der: bytes, contents: bytes | None) -> (VerifiedClient, bytes). This extracts the signature, optionally contents, and also certificates from teh PKCS#7 structure. It then performs trust building (using the provided verifier), and finally verifies the signature. It returns the cert (and chain, and valid subjects) that was ultimately used, as well as the contents. Users can control what X.509 path building parameters, what roots, etc. on the verifier they themselves construct.

We think these two APIs provide everything you need to do useful real world PKCS#7 verification -- SMIME and PEM become very simple compositions on top of them. We believe each of these can be landed independently (verify_pkcs7_directly_signed_by first, of course).

If folks on this thread are willing, we'd love feedback on whether you think this works for your use cases and can be used securely!

TaaviE · 2026-06-10T09:23:16Z

[...] (optionally) contents from the PKCS#7 structure

This could be a dangerous construction. Signature should be verified against something specific and explicit. If contents are passed then bytes should not be returned (nothing should be extracted) or vice versa. People have shot themselves in the foot before with this and I think this current PR does the same.

verything you need to do useful real world PKCS#7 verification -- SMIME and PEM become very simple compositions on top of them

Please keep in mind that if it's PKCS#7, you can only support really old S/MIME. New S/MIME is built on top of CMS. CMS is required for S/MIME to support ECDSA, EdDSA, ML-DSA, AES-GCM, OCSP, etc.

nitneuqr changed the title ~~Add support for decrypting S/MIME messages~~ Add support for verifying S/MIME messages Jan 10, 2025

nitneuqr commented Jan 10, 2025

View reviewed changes

Tarnum-tst reviewed Jan 11, 2025

View reviewed changes

Comment thread src/rust/src/pkcs7.rs

Tarnum-tst approved these changes Jan 13, 2025

View reviewed changes

nitneuqr force-pushed the pkcs7-verify branch from 5f06262 to f2f9be4 Compare January 15, 2025 09:20

nitneuqr requested a review from alex January 20, 2025 15:12

alex mentioned this pull request Jan 28, 2025

support for PKCS7_NO_VERIFY in test_support.pkcs7_verify #12116

Closed

alex reviewed Jan 31, 2025

View reviewed changes

lgcCerti mentioned this pull request Feb 6, 2025

Verifying PKCS7 CMS message with hazmat OpenSSL Bindings #12406

Closed

nitneuqr force-pushed the pkcs7-verify branch from fb78777 to c45ee63 Compare February 15, 2025 10:34

nitneuqr mentioned this pull request Feb 15, 2025

PKCS#7 signing & verification - Certificate extension policies #12465

Closed

nitneuqr force-pushed the pkcs7-verify branch from aa7b39f to 412c614 Compare June 15, 2025 10:08

nitneuqr added 2 commits September 13, 2025 18:28

feat: PKCS#7 extension policies

db7eab9

added tests accordingly adapted the pkcs7 certificate adapted EE policy do not know if a CA policy is needed! added SAN checking

adapted based on comments & prauscher feedback

ceb8b2a

nitneuqr force-pushed the pkcs7-verify branch from 8975aa9 to 206fe96 Compare September 13, 2025 17:22

nitneuqr force-pushed the pkcs7-verify branch from ab7b125 to e6ddf30 Compare September 13, 2025 17:32

BeryJu mentioned this pull request Dec 28, 2025

Automatically generated certificates are not self-signed goauthentik/authentik#19058

Closed

Tarnum-tst reviewed Jan 9, 2026

View reviewed changes

TaaviE mentioned this pull request Jan 22, 2026

Add CABF S/MIME certificate policies #14190

Closed

cluck mentioned this pull request Jan 30, 2026

Consider porting to Cryptography cluck/rdpsign#3

Open

mattmccutchen mentioned this pull request May 24, 2026

qubes-fwupdmgr refresh fails because firmware.xml.zst.jcat no longer contains a PGP signature QubesOS/qubes-issues#10869

Open

		let certificates = pyo3::types::PyList::empty(py);
		certificates.append(certificate)?;

Conversation

nitneuqr commented Jan 10, 2025

Uh oh!

alex commented Jan 10, 2025 via email

Uh oh!

nitneuqr Jan 10, 2025

Choose a reason for hiding this comment

Uh oh!

nitneuqr commented Jan 11, 2025

Uh oh!

Uh oh!

alex commented Jan 13, 2025

Uh oh!

woodruffw commented Jan 13, 2025

Uh oh!

nitneuqr commented Jan 13, 2025

Uh oh!

nitneuqr commented Jan 15, 2025

Uh oh!

nitneuqr commented Jan 15, 2025

Uh oh!

alex commented Jan 20, 2025 via email

Uh oh!

nitneuqr commented Jan 21, 2025

Uh oh!

prauscher commented Jan 24, 2025

Uh oh!

alex commented Jan 31, 2025

Uh oh!

Uh oh!

Uh oh!

alex Jan 31, 2025

Choose a reason for hiding this comment

Uh oh!

nitneuqr commented Feb 2, 2025

Uh oh!

lgcCerti commented Feb 6, 2025

Uh oh!

alex commented Feb 14, 2025

Uh oh!

nitneuqr commented Feb 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nitneuqr commented Feb 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alex commented Feb 15, 2025 via email

Uh oh!

nitneuqr commented Jun 15, 2025

Uh oh!

Tarnum-tst Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

TaaviE commented Feb 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alex commented Jun 9, 2026

Uh oh!

TaaviE commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

7 participants

nitneuqr commented Feb 15, 2025 •

edited

Loading

nitneuqr commented Feb 15, 2025 •

edited

Loading

TaaviE commented Feb 13, 2026 •

edited

Loading

TaaviE commented Jun 10, 2026 •

edited

Loading