The blog post provides a certificate chain that validates in OpenSSL but not in Go.
The reason it doesn't validate in Go is that the Subject field in the CA certificate uses a different string encoding than the Issuer field in the leaf certificate, so the fields are not byte-for-byte equal.
Go requires the Issuer and Subject to be byte-for-byte equal. This was permitted by older specs, but RFC 5280 changed the rules to require the use of RFC 4518 (LDAP stringprep) for comparing strings. This turned a simple memcmp into a complicated algorithm that requires implementing Unicode normalization, for virtually zero benefit. That's the last thing you want in your security-critical certificate verifier, so Go quite sensibly chose to follow the older specs in this regard. The CA/Browser Forum's Baseline Requirements also mandate byte-for-byte equality, so Go's behavior won't cause publicly-trusted certificates to be incorrectly rejected.
Note that LDAP stringprep is so complicated that OpenSSL doesn't even try to implement it properly and uses an approximation instead. So you would also be able to "fool" OpenSSL into rejecting certificate chains that RFC 5280 says are valid.
The blog post says that this is an "ongoing debate" in the Go project but I don't think that's accurate. I'd be shocked if they ever changed this behavior, given that crypto/x509 targets publicly-trusted certificates and the current behavior is so much simpler.
Something I noticed decades ago is that some small, innocuous features can drag with them giant ecosystems of software.
I first noticed this when I had to implement a C++ client for a custom RPC protocol and the dev “on the other end of the wire” added one new “convenience” in the data types supported… which would have required me to include the entire Java runtime in my client!
All protocol specs are vulnerable to this effect where it’s all too easy to require clients to include half a dozen different regex engines, three byte code virtual machines, and most of LLVM for good measure.
This is uninteresting. CAs are well aware that they have to encode the subject DN and issuer DN identically to maximize interoperability. There are several implementations that require that.
If we were to make a new version of the spec for X.509 certificates, I would hope that we would eliminate all the non-UTF8 encodings so that this would be a non-issue.
This is what happens when kids today are unaware of history. This was a known problem 30+ years ago, and the Go kids have just rediscovered it for themselves. The most extreme case of this madness was imagining you could re-encode certificates into a DER blob from their stored components and the signature would still validate, something that OER (from memory) guys are now trying to do.
The rules for DNs are "there is only one encoding rule and that is memcpy(); there is only one matching rule and that is memcmp()". Given that Go has fallen into the decades-old trap of trying to re-encode strings, it's bound to be vulnerable to any number of other issues like evading excludedSubtrees through string-encoding tricks.
Between this and the IPv6 zone identifier issue, it feels like there's a bit of a trend of commenters more or less assuming Go is doing the wrong thing when it's actually following the standards/best practices more correctly than average. I wonder where this reputation came from.
From the article, it doesn't seem like Go is trying to re-encode strings? Go is saying (correctly, IMO) that a UTF8String field in the Issuer is not the same as a PrintableString field in the Subject.
Ah, you're right, I was a bit confused by the bouncing back and forth between Go and OpenSSL and the title, "Fooling Go's X.509" when in fact on re-read Go appears to be doing the right thing and using a strict compare while OpenSSL uses the open-to-manipulation compare.
The blog post provides a certificate chain that validates in OpenSSL but not in Go.
The reason it doesn't validate in Go is that the Subject field in the CA certificate uses a different string encoding than the Issuer field in the leaf certificate, so the fields are not byte-for-byte equal.
Go requires the Issuer and Subject to be byte-for-byte equal. This was permitted by older specs, but RFC 5280 changed the rules to require the use of RFC 4518 (LDAP stringprep) for comparing strings. This turned a simple memcmp into a complicated algorithm that requires implementing Unicode normalization, for virtually zero benefit. That's the last thing you want in your security-critical certificate verifier, so Go quite sensibly chose to follow the older specs in this regard. The CA/Browser Forum's Baseline Requirements also mandate byte-for-byte equality, so Go's behavior won't cause publicly-trusted certificates to be incorrectly rejected.
Note that LDAP stringprep is so complicated that OpenSSL doesn't even try to implement it properly and uses an approximation instead. So you would also be able to "fool" OpenSSL into rejecting certificate chains that RFC 5280 says are valid.
The blog post says that this is an "ongoing debate" in the Go project but I don't think that's accurate. I'd be shocked if they ever changed this behavior, given that crypto/x509 targets publicly-trusted certificates and the current behavior is so much simpler.
I feel like basically all the X509 threads on HN should basically be locked until after you write your first comment on them.
Aw, thanks :-)
Yeah, no, Andrew has insightful things to say, but he's hardly indispensable.
Something I noticed decades ago is that some small, innocuous features can drag with them giant ecosystems of software.
I first noticed this when I had to implement a C++ client for a custom RPC protocol and the dev “on the other end of the wire” added one new “convenience” in the data types supported… which would have required me to include the entire Java runtime in my client!
All protocol specs are vulnerable to this effect where it’s all too easy to require clients to include half a dozen different regex engines, three byte code virtual machines, and most of LLVM for good measure.
any specification eventually grows to encompass all features of its original implementation language.
This is uninteresting. CAs are well aware that they have to encode the subject DN and issuer DN identically to maximize interoperability. There are several implementations that require that.
If we were to make a new version of the spec for X.509 certificates, I would hope that we would eliminate all the non-UTF8 encodings so that this would be a non-issue.
I don't think that's an ongoing debate looks to me that it ended in 2019 https://github.com/golang/go/issues/31440#issuecomment-53724...
This is what happens when kids today are unaware of history. This was a known problem 30+ years ago, and the Go kids have just rediscovered it for themselves. The most extreme case of this madness was imagining you could re-encode certificates into a DER blob from their stored components and the signature would still validate, something that OER (from memory) guys are now trying to do.
The rules for DNs are "there is only one encoding rule and that is memcpy(); there is only one matching rule and that is memcmp()". Given that Go has fallen into the decades-old trap of trying to re-encode strings, it's bound to be vulnerable to any number of other issues like evading excludedSubtrees through string-encoding tricks.
Between this and the IPv6 zone identifier issue, it feels like there's a bit of a trend of commenters more or less assuming Go is doing the wrong thing when it's actually following the standards/best practices more correctly than average. I wonder where this reputation came from.
Most people accessing a site are likely not using Golang and are using Chrome. Thus Chrome is assumed right and Go is the one that's the outlier.
From the article, it doesn't seem like Go is trying to re-encode strings? Go is saying (correctly, IMO) that a UTF8String field in the Issuer is not the same as a PrintableString field in the Subject.
Ah, you're right, I was a bit confused by the bouncing back and forth between Go and OpenSSL and the title, "Fooling Go's X.509" when in fact on re-read Go appears to be doing the right thing and using a strict compare while OpenSSL uses the open-to-manipulation compare.
The Go "kids" are famous for, among other things, being industry leaders 30 years ago.
Those young whipper snappers with their lifetime of experience!