Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better TLS certificate validation #1851

Closed
ridercz opened this issue Apr 3, 2016 · 37 comments
Closed

Better TLS certificate validation #1851

ridercz opened this issue Apr 3, 2016 · 37 comments

Comments

@ridercz
Copy link

ridercz commented Apr 3, 2016

It's great that this project supports HTTPS/TLS, but the current implementation is very close to worthless from security standpoint. Because when I can check only direct certificate fingerprint, it means that the certificate basically cannot be ever changed - when expires or is compromised.

I fully understand it's not possible to add full validation of certificate chain, but maybe it would be possible to add one-level validation to the CA? It would change the problem from "you can't change the server certificate, ever" to "you can't change the CA". Which is still far from ideal, but can actually provide some manageable security?

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

@igrr
Copy link
Member

igrr commented Apr 3, 2016

Validation of CA certificate is easy to add, but this alone doesn't let you deal with compromised certificates. You also need to support CRL/OCSP.
Until som

@bbx10
Copy link
Contributor

bbx10 commented Apr 25, 2016

There is a good reason to check the cert chain instead of server cert fingerprint. Some server certs expire every 90 days so that means the ESP program must be updated with a new fingerprint every 90 days or sooner. Root CA certs expiration dates are usually many years in the future so they only rarely need to be updated.

https://letsencrypt.org/2015/11/09/why-90-days.html

@Potato-Matic
Copy link

Side note: is storage is an issue, could the internal FS be used to store certs and such? Some modules have a rather large flash...

@Potato-Matic
Copy link

One issue I've found is with having my ESP8266 interact with various google APIs. I supply the correct fingerprint in the sketch, but it doesn't take long before the sketch starts to fail and eventually doesn't work at all, as google transitions to another certificate. (they have their own intermediate CA and can issue themselves as many as they like)

If it were possible to look up the chain a level at the CA to see if the certificate is valid it would be great, so that I don't have to keep changing the certificate fingerprint in my sketch. Alternatively, if there's a way to get an ESP8266 to just look for the new fingerprint or ignore it completely, that would be great. I have to use https, as it's the only way google's APIs will allow a connection, but can't seem to find a way around the frequently changing fingerprint causing things to fail.

@Potato-Matic
Copy link

After a fair length of testing, I've found that the cert(s) used by the outward facing interface of google scripts change about every week and a half, meaning I frequently have to patch my code with the new fp.

@igrr
Copy link
Member

igrr commented Aug 25, 2016

I got this working, will put up a pull request soon.

@igrr
Copy link
Member

igrr commented Aug 25, 2016

Pushed to master.

@Potato-Matic
Copy link

igrr, you've done absolute miracles, once again

Some day I should like to buy you a beer or something.

@jorgerobles
Copy link

Is there any full example of working with root cert? Thanks!

@Potato-Matic
Copy link

If this is pushed to master, and assuming I've updated my clone of the git, is it now possible to handle https via higher-level certs? If so, has anyone figured out how to make it work?

@Potato-Matic
Copy link

If this is master now, does that mean it's available on an up-to-date install? Does anyone have any example for how to do this? Can I just provide the cert fingerprint of a higher level CA? If it checks against CAs, do I have to provide a fingerprint at all?

@ghost
Copy link

ghost commented Oct 29, 2016

I am also curious how to find the source for this change.

@aykevl
Copy link

aykevl commented Nov 13, 2016

This commit? It doesn't seem to help much for HTTPClient.
b412660

@alancinacio
Copy link

alancinacio commented Nov 13, 2016

Thank you @igrr ! Do you have an example of how to use the verifyCertChain?

@rakeshpai
Copy link

Just discovered this issue, and it's a Very Big Deal (TM) if this is working. Thanks, @igrr. Would love to see an example of its usage.

@gjt211
Copy link

gjt211 commented Dec 2, 2016

Is there any updates on this yet?

@Potato-Matic
Copy link

What's the current status of this? Can I just use the fingerprint of a cert higher in the chain, and it will just happen automagically? Or is there some additional steps or considerations to follow?

@robfullagar
Copy link

Has the issue of not allowing the client cert and key to be loaded until a connection is made been resolved yet?
I am wanting to send the client cert to the serve to mutually authenticate the connection when the TLS session is created.

Cheers

Rob

@igrr
Copy link
Member

igrr commented May 20, 2017

Has the issue of not allowing the client cert and key to be loaded until a connection is made been resolved yet?

Do you have a description of this issue?

@robfullagar
Copy link

Using loadcertificate or loadprivatekey fails until connect has been called and a connection has been established. My server requests the client cert as part of the tls handshake but is empty as I can't load it. I think is is related to or called the axtls regression bug?

@igrr
Copy link
Member

igrr commented May 21, 2017

Thanks for reminding me about that one, it's been reported in #2470. Not fixed yet but I will take a look at it this week.

@igrr
Copy link
Member

igrr commented May 21, 2017

Example of certificate verification using root CA is added in PR #3271.

@robfullagar
Copy link

Igrr

This works great for me now...client certs are being oassed to the server as part of the TLS handshake.

Cheers

Rob

@ilg
Copy link

ilg commented Jun 9, 2017

Using 2.4.0-rc1, I'm able to get the example added in #3271 to work, but things go sideways when I try to validate for a subdomain of appspot.com. The certificate chain for *.appspot.com is:

GeoTrust Global CA > Google Internet Authority G2 > *.appspot.com

Using the GeoTrust Global CA certificate as the CACert, I get:

ssl_verify_cert returned -523

which seems to be due to the GeoTrust Global CA certificate having Basic Constraint CA true, but not having a Key Usage extension (https://github.com/igrr/axtls-8266/blob/master/ssl/x509.c#L499-L511, I think). Is there something I'm missing here that should make this work?

In an attempt to work around that, I tried using the Google Internet Authority G2 certificate as the CACert, but got the same error—though that certificate appears to have the Basic Constraint CA set to true, have a Key Usage extension, have Key Cert Sign set in that extension, and the chain should satisfy the path length constraint, so I'm not clear on why that's failing, either.

For reference, here are my conversions of those certificates to unsigned char[]:

GeoTrust Global CA:

const unsigned char caCert[] = {
  0x30, 0x82, 0x03, 0x54, 0x30, 0x82, 0x02, 0x3c, 0xa0, 0x03, 0x02, 0x01, 0x02, 0x02, 0x03, 0x02,
  0x34, 0x56, 0x30, 0x0d, 0x06, 0x09, 0x2a, 0x86, 0x48, 0x86, 0xf7, 0x0d, 0x01, 0x01, 0x05, 0x05,
  0x00, 0x30, 0x42, 0x31, 0x0b, 0x30, 0x09, 0x06, 0x03, 0x55, 0x04, 0x06, 0x13, 0x02, 0x55, 0x53,
  0x31, 0x16, 0x30, 0x14, 0x06, 0x03, 0x55, 0x04, 0x0a, 0x13, 0x0d, 0x47, 0x65, 0x6f, 0x54, 0x72,
  0x75, 0x73, 0x74, 0x20, 0x49, 0x6e, 0x63, 0x2e, 0x31, 0x1b, 0x30, 0x19, 0x06, 0x03, 0x55, 0x04,
  0x03, 0x13, 0x12, 0x47, 0x65, 0x6f, 0x54, 0x72, 0x75, 0x73, 0x74, 0x20, 0x47, 0x6c, 0x6f, 0x62,
  0x61, 0x6c, 0x20, 0x43, 0x41, 0x30, 0x1e, 0x17, 0x0d, 0x30, 0x32, 0x30, 0x35, 0x32, 0x31, 0x30,
  0x34, 0x30, 0x30, 0x30, 0x30, 0x5a, 0x17, 0x0d, 0x32, 0x32, 0x30, 0x35, 0x32, 0x31, 0x30, 0x34,
  0x30, 0x30, 0x30, 0x30, 0x5a, 0x30, 0x42, 0x31, 0x0b, 0x30, 0x09, 0x06, 0x03, 0x55, 0x04, 0x06,
  0x13, 0x02, 0x55, 0x53, 0x31, 0x16, 0x30, 0x14, 0x06, 0x03, 0x55, 0x04, 0x0a, 0x13, 0x0d, 0x47,
  0x65, 0x6f, 0x54, 0x72, 0x75, 0x73, 0x74, 0x20, 0x49, 0x6e, 0x63, 0x2e, 0x31, 0x1b, 0x30, 0x19,
  0x06, 0x03, 0x55, 0x04, 0x03, 0x13, 0x12, 0x47, 0x65, 0x6f, 0x54, 0x72, 0x75, 0x73, 0x74, 0x20,
  0x47, 0x6c, 0x6f, 0x62, 0x61, 0x6c, 0x20, 0x43, 0x41, 0x30, 0x82, 0x01, 0x22, 0x30, 0x0d, 0x06,
  0x09, 0x2a, 0x86, 0x48, 0x86, 0xf7, 0x0d, 0x01, 0x01, 0x01, 0x05, 0x00, 0x03, 0x82, 0x01, 0x0f,
  0x00, 0x30, 0x82, 0x01, 0x0a, 0x02, 0x82, 0x01, 0x01, 0x00, 0xda, 0xcc, 0x18, 0x63, 0x30, 0xfd,
  0xf4, 0x17, 0x23, 0x1a, 0x56, 0x7e, 0x5b, 0xdf, 0x3c, 0x6c, 0x38, 0xe4, 0x71, 0xb7, 0x78, 0x91,
  0xd4, 0xbc, 0xa1, 0xd8, 0x4c, 0xf8, 0xa8, 0x43, 0xb6, 0x03, 0xe9, 0x4d, 0x21, 0x07, 0x08, 0x88,
  0xda, 0x58, 0x2f, 0x66, 0x39, 0x29, 0xbd, 0x05, 0x78, 0x8b, 0x9d, 0x38, 0xe8, 0x05, 0xb7, 0x6a,
  0x7e, 0x71, 0xa4, 0xe6, 0xc4, 0x60, 0xa6, 0xb0, 0xef, 0x80, 0xe4, 0x89, 0x28, 0x0f, 0x9e, 0x25,
  0xd6, 0xed, 0x83, 0xf3, 0xad, 0xa6, 0x91, 0xc7, 0x98, 0xc9, 0x42, 0x18, 0x35, 0x14, 0x9d, 0xad,
  0x98, 0x46, 0x92, 0x2e, 0x4f, 0xca, 0xf1, 0x87, 0x43, 0xc1, 0x16, 0x95, 0x57, 0x2d, 0x50, 0xef,
  0x89, 0x2d, 0x80, 0x7a, 0x57, 0xad, 0xf2, 0xee, 0x5f, 0x6b, 0xd2, 0x00, 0x8d, 0xb9, 0x14, 0xf8,
  0x14, 0x15, 0x35, 0xd9, 0xc0, 0x46, 0xa3, 0x7b, 0x72, 0xc8, 0x91, 0xbf, 0xc9, 0x55, 0x2b, 0xcd,
  0xd0, 0x97, 0x3e, 0x9c, 0x26, 0x64, 0xcc, 0xdf, 0xce, 0x83, 0x19, 0x71, 0xca, 0x4e, 0xe6, 0xd4,
  0xd5, 0x7b, 0xa9, 0x19, 0xcd, 0x55, 0xde, 0xc8, 0xec, 0xd2, 0x5e, 0x38, 0x53, 0xe5, 0x5c, 0x4f,
  0x8c, 0x2d, 0xfe, 0x50, 0x23, 0x36, 0xfc, 0x66, 0xe6, 0xcb, 0x8e, 0xa4, 0x39, 0x19, 0x00, 0xb7,
  0x95, 0x02, 0x39, 0x91, 0x0b, 0x0e, 0xfe, 0x38, 0x2e, 0xd1, 0x1d, 0x05, 0x9a, 0xf6, 0x4d, 0x3e,
  0x6f, 0x0f, 0x07, 0x1d, 0xaf, 0x2c, 0x1e, 0x8f, 0x60, 0x39, 0xe2, 0xfa, 0x36, 0x53, 0x13, 0x39,
  0xd4, 0x5e, 0x26, 0x2b, 0xdb, 0x3d, 0xa8, 0x14, 0xbd, 0x32, 0xeb, 0x18, 0x03, 0x28, 0x52, 0x04,
  0x71, 0xe5, 0xab, 0x33, 0x3d, 0xe1, 0x38, 0xbb, 0x07, 0x36, 0x84, 0x62, 0x9c, 0x79, 0xea, 0x16,
  0x30, 0xf4, 0x5f, 0xc0, 0x2b, 0xe8, 0x71, 0x6b, 0xe4, 0xf9, 0x02, 0x03, 0x01, 0x00, 0x01, 0xa3,
  0x53, 0x30, 0x51, 0x30, 0x0f, 0x06, 0x03, 0x55, 0x1d, 0x13, 0x01, 0x01, 0xff, 0x04, 0x05, 0x30,
  0x03, 0x01, 0x01, 0xff, 0x30, 0x1d, 0x06, 0x03, 0x55, 0x1d, 0x0e, 0x04, 0x16, 0x04, 0x14, 0xc0,
  0x7a, 0x98, 0x68, 0x8d, 0x89, 0xfb, 0xab, 0x05, 0x64, 0x0c, 0x11, 0x7d, 0xaa, 0x7d, 0x65, 0xb8,
  0xca, 0xcc, 0x4e, 0x30, 0x1f, 0x06, 0x03, 0x55, 0x1d, 0x23, 0x04, 0x18, 0x30, 0x16, 0x80, 0x14,
  0xc0, 0x7a, 0x98, 0x68, 0x8d, 0x89, 0xfb, 0xab, 0x05, 0x64, 0x0c, 0x11, 0x7d, 0xaa, 0x7d, 0x65,
  0xb8, 0xca, 0xcc, 0x4e, 0x30, 0x0d, 0x06, 0x09, 0x2a, 0x86, 0x48, 0x86, 0xf7, 0x0d, 0x01, 0x01,
  0x05, 0x05, 0x00, 0x03, 0x82, 0x01, 0x01, 0x00, 0x35, 0xe3, 0x29, 0x6a, 0xe5, 0x2f, 0x5d, 0x54,
  0x8e, 0x29, 0x50, 0x94, 0x9f, 0x99, 0x1a, 0x14, 0xe4, 0x8f, 0x78, 0x2a, 0x62, 0x94, 0xa2, 0x27,
  0x67, 0x9e, 0xd0, 0xcf, 0x1a, 0x5e, 0x47, 0xe9, 0xc1, 0xb2, 0xa4, 0xcf, 0xdd, 0x41, 0x1a, 0x05,
  0x4e, 0x9b, 0x4b, 0xee, 0x4a, 0x6f, 0x55, 0x52, 0xb3, 0x24, 0xa1, 0x37, 0x0a, 0xeb, 0x64, 0x76,
  0x2a, 0x2e, 0x2c, 0xf3, 0xfd, 0x3b, 0x75, 0x90, 0xbf, 0xfa, 0x71, 0xd8, 0xc7, 0x3d, 0x37, 0xd2,
  0xb5, 0x05, 0x95, 0x62, 0xb9, 0xa6, 0xde, 0x89, 0x3d, 0x36, 0x7b, 0x38, 0x77, 0x48, 0x97, 0xac,
  0xa6, 0x20, 0x8f, 0x2e, 0xa6, 0xc9, 0x0c, 0xc2, 0xb2, 0x99, 0x45, 0x00, 0xc7, 0xce, 0x11, 0x51,
  0x22, 0x22, 0xe0, 0xa5, 0xea, 0xb6, 0x15, 0x48, 0x09, 0x64, 0xea, 0x5e, 0x4f, 0x74, 0xf7, 0x05,
  0x3e, 0xc7, 0x8a, 0x52, 0x0c, 0xdb, 0x15, 0xb4, 0xbd, 0x6d, 0x9b, 0xe5, 0xc6, 0xb1, 0x54, 0x68,
  0xa9, 0xe3, 0x69, 0x90, 0xb6, 0x9a, 0xa5, 0x0f, 0xb8, 0xb9, 0x3f, 0x20, 0x7d, 0xae, 0x4a, 0xb5,
  0xb8, 0x9c, 0xe4, 0x1d, 0xb6, 0xab, 0xe6, 0x94, 0xa5, 0xc1, 0xc7, 0x83, 0xad, 0xdb, 0xf5, 0x27,
  0x87, 0x0e, 0x04, 0x6c, 0xd5, 0xff, 0xdd, 0xa0, 0x5d, 0xed, 0x87, 0x52, 0xb7, 0x2b, 0x15, 0x02,
  0xae, 0x39, 0xa6, 0x6a, 0x74, 0xe9, 0xda, 0xc4, 0xe7, 0xbc, 0x4d, 0x34, 0x1e, 0xa9, 0x5c, 0x4d,
  0x33, 0x5f, 0x92, 0x09, 0x2f, 0x88, 0x66, 0x5d, 0x77, 0x97, 0xc7, 0x1d, 0x76, 0x13, 0xa9, 0xd5,
  0xe5, 0xf1, 0x16, 0x09, 0x11, 0x35, 0xd5, 0xac, 0xdb, 0x24, 0x71, 0x70, 0x2c, 0x98, 0x56, 0x0b,
  0xd9, 0x17, 0xb4, 0xd1, 0xe3, 0x51, 0x2b, 0x5e, 0x75, 0xe8, 0xd5, 0xd0, 0xdc, 0x4f, 0x34, 0xed,
  0xc2, 0x05, 0x66, 0x80, 0xa1, 0xcb, 0xe6, 0x33
};
const unsigned int caCertLen = 856;

Google Internet Authority G2:

const unsigned char caCert[] = {
  0x30, 0x82, 0x04, 0x28, 0x30, 0x82, 0x03, 0x10, 0xa0, 0x03, 0x02, 0x01, 0x02, 0x02, 0x10, 0x01,
  0x00, 0x21, 0x25, 0x88, 0xb0, 0xfa, 0x59, 0xa7, 0x77, 0xef, 0x05, 0x7b, 0x66, 0x27, 0xdf, 0x30,
  0x0d, 0x06, 0x09, 0x2a, 0x86, 0x48, 0x86, 0xf7, 0x0d, 0x01, 0x01, 0x0b, 0x05, 0x00, 0x30, 0x42,
  0x31, 0x0b, 0x30, 0x09, 0x06, 0x03, 0x55, 0x04, 0x06, 0x13, 0x02, 0x55, 0x53, 0x31, 0x16, 0x30,
  0x14, 0x06, 0x03, 0x55, 0x04, 0x0a, 0x13, 0x0d, 0x47, 0x65, 0x6f, 0x54, 0x72, 0x75, 0x73, 0x74,
  0x20, 0x49, 0x6e, 0x63, 0x2e, 0x31, 0x1b, 0x30, 0x19, 0x06, 0x03, 0x55, 0x04, 0x03, 0x13, 0x12,
  0x47, 0x65, 0x6f, 0x54, 0x72, 0x75, 0x73, 0x74, 0x20, 0x47, 0x6c, 0x6f, 0x62, 0x61, 0x6c, 0x20,
  0x43, 0x41, 0x30, 0x1e, 0x17, 0x0d, 0x31, 0x37, 0x30, 0x35, 0x32, 0x32, 0x31, 0x31, 0x33, 0x32,
  0x33, 0x37, 0x5a, 0x17, 0x0d, 0x31, 0x38, 0x31, 0x32, 0x33, 0x31, 0x32, 0x33, 0x35, 0x39, 0x35,
  0x39, 0x5a, 0x30, 0x49, 0x31, 0x0b, 0x30, 0x09, 0x06, 0x03, 0x55, 0x04, 0x06, 0x13, 0x02, 0x55,
  0x53, 0x31, 0x13, 0x30, 0x11, 0x06, 0x03, 0x55, 0x04, 0x0a, 0x13, 0x0a, 0x47, 0x6f, 0x6f, 0x67,
  0x6c, 0x65, 0x20, 0x49, 0x6e, 0x63, 0x31, 0x25, 0x30, 0x23, 0x06, 0x03, 0x55, 0x04, 0x03, 0x13,
  0x1c, 0x47, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x20, 0x49, 0x6e, 0x74, 0x65, 0x72, 0x6e, 0x65, 0x74,
  0x20, 0x41, 0x75, 0x74, 0x68, 0x6f, 0x72, 0x69, 0x74, 0x79, 0x20, 0x47, 0x32, 0x30, 0x82, 0x01,
  0x22, 0x30, 0x0d, 0x06, 0x09, 0x2a, 0x86, 0x48, 0x86, 0xf7, 0x0d, 0x01, 0x01, 0x01, 0x05, 0x00,
  0x03, 0x82, 0x01, 0x0f, 0x00, 0x30, 0x82, 0x01, 0x0a, 0x02, 0x82, 0x01, 0x01, 0x00, 0x9c, 0x2a,
  0x04, 0x77, 0x5c, 0xd8, 0x50, 0x91, 0x3a, 0x06, 0xa3, 0x82, 0xe0, 0xd8, 0x50, 0x48, 0xbc, 0x89,
  0x3f, 0xf1, 0x19, 0x70, 0x1a, 0x88, 0x46, 0x7e, 0xe0, 0x8f, 0xc5, 0xf1, 0x89, 0xce, 0x21, 0xee,
  0x5a, 0xfe, 0x61, 0x0d, 0xb7, 0x32, 0x44, 0x89, 0xa0, 0x74, 0x0b, 0x53, 0x4f, 0x55, 0xa4, 0xce,
  0x82, 0x62, 0x95, 0xee, 0xeb, 0x59, 0x5f, 0xc6, 0xe1, 0x05, 0x80, 0x12, 0xc4, 0x5e, 0x94, 0x3f,
  0xbc, 0x5b, 0x48, 0x38, 0xf4, 0x53, 0xf7, 0x24, 0xe6, 0xfb, 0x91, 0xe9, 0x15, 0xc4, 0xcf, 0xf4,
  0x53, 0x0d, 0xf4, 0x4a, 0xfc, 0x9f, 0x54, 0xde, 0x7d, 0xbe, 0xa0, 0x6b, 0x6f, 0x87, 0xc0, 0xd0,
  0x50, 0x1f, 0x28, 0x30, 0x03, 0x40, 0xda, 0x08, 0x73, 0x51, 0x6c, 0x7f, 0xff, 0x3a, 0x3c, 0xa7,
  0x37, 0x06, 0x8e, 0xbd, 0x4b, 0x11, 0x04, 0xeb, 0x7d, 0x24, 0xde, 0xe6, 0xf9, 0xfc, 0x31, 0x71,
  0xfb, 0x94, 0xd5, 0x60, 0xf3, 0x2e, 0x4a, 0xaf, 0x42, 0xd2, 0xcb, 0xea, 0xc4, 0x6a, 0x1a, 0xb2,
  0xcc, 0x53, 0xdd, 0x15, 0x4b, 0x8b, 0x1f, 0xc8, 0x19, 0x61, 0x1f, 0xcd, 0x9d, 0xa8, 0x3e, 0x63,
  0x2b, 0x84, 0x35, 0x69, 0x65, 0x84, 0xc8, 0x19, 0xc5, 0x46, 0x22, 0xf8, 0x53, 0x95, 0xbe, 0xe3,
  0x80, 0x4a, 0x10, 0xc6, 0x2a, 0xec, 0xba, 0x97, 0x20, 0x11, 0xc7, 0x39, 0x99, 0x10, 0x04, 0xa0,
  0xf0, 0x61, 0x7a, 0x95, 0x25, 0x8c, 0x4e, 0x52, 0x75, 0xe2, 0xb6, 0xed, 0x08, 0xca, 0x14, 0xfc,
  0xce, 0x22, 0x6a, 0xb3, 0x4e, 0xcf, 0x46, 0x03, 0x97, 0x97, 0x03, 0x7e, 0xc0, 0xb1, 0xde, 0x7b,
  0xaf, 0x45, 0x33, 0xcf, 0xba, 0x3e, 0x71, 0xb7, 0xde, 0xf4, 0x25, 0x25, 0xc2, 0x0d, 0x35, 0x89,
  0x9d, 0x9d, 0xfb, 0x0e, 0x11, 0x79, 0x89, 0x1e, 0x37, 0xc5, 0xaf, 0x8e, 0x72, 0x69, 0x02, 0x03,
  0x01, 0x00, 0x01, 0xa3, 0x82, 0x01, 0x11, 0x30, 0x82, 0x01, 0x0d, 0x30, 0x1f, 0x06, 0x03, 0x55,
  0x1d, 0x23, 0x04, 0x18, 0x30, 0x16, 0x80, 0x14, 0xc0, 0x7a, 0x98, 0x68, 0x8d, 0x89, 0xfb, 0xab,
  0x05, 0x64, 0x0c, 0x11, 0x7d, 0xaa, 0x7d, 0x65, 0xb8, 0xca, 0xcc, 0x4e, 0x30, 0x1d, 0x06, 0x03,
  0x55, 0x1d, 0x0e, 0x04, 0x16, 0x04, 0x14, 0x4a, 0xdd, 0x06, 0x16, 0x1b, 0xbc, 0xf6, 0x68, 0xb5,
  0x76, 0xf5, 0x81, 0xb6, 0xbb, 0x62, 0x1a, 0xba, 0x5a, 0x81, 0x2f, 0x30, 0x0e, 0x06, 0x03, 0x55,
  0x1d, 0x0f, 0x01, 0x01, 0xff, 0x04, 0x04, 0x03, 0x02, 0x01, 0x06, 0x30, 0x2e, 0x06, 0x08, 0x2b,
  0x06, 0x01, 0x05, 0x05, 0x07, 0x01, 0x01, 0x04, 0x22, 0x30, 0x20, 0x30, 0x1e, 0x06, 0x08, 0x2b,
  0x06, 0x01, 0x05, 0x05, 0x07, 0x30, 0x01, 0x86, 0x12, 0x68, 0x74, 0x74, 0x70, 0x3a, 0x2f, 0x2f,
  0x67, 0x2e, 0x73, 0x79, 0x6d, 0x63, 0x64, 0x2e, 0x63, 0x6f, 0x6d, 0x30, 0x12, 0x06, 0x03, 0x55,
  0x1d, 0x13, 0x01, 0x01, 0xff, 0x04, 0x08, 0x30, 0x06, 0x01, 0x01, 0xff, 0x02, 0x01, 0x00, 0x30,
  0x35, 0x06, 0x03, 0x55, 0x1d, 0x1f, 0x04, 0x2e, 0x30, 0x2c, 0x30, 0x2a, 0xa0, 0x28, 0xa0, 0x26,
  0x86, 0x24, 0x68, 0x74, 0x74, 0x70, 0x3a, 0x2f, 0x2f, 0x67, 0x2e, 0x73, 0x79, 0x6d, 0x63, 0x62,
  0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x63, 0x72, 0x6c, 0x73, 0x2f, 0x67, 0x74, 0x67, 0x6c, 0x6f, 0x62,
  0x61, 0x6c, 0x2e, 0x63, 0x72, 0x6c, 0x30, 0x21, 0x06, 0x03, 0x55, 0x1d, 0x20, 0x04, 0x1a, 0x30,
  0x18, 0x30, 0x0c, 0x06, 0x0a, 0x2b, 0x06, 0x01, 0x04, 0x01, 0xd6, 0x79, 0x02, 0x05, 0x01, 0x30,
  0x08, 0x06, 0x06, 0x67, 0x81, 0x0c, 0x01, 0x02, 0x02, 0x30, 0x1d, 0x06, 0x03, 0x55, 0x1d, 0x25,
  0x04, 0x16, 0x30, 0x14, 0x06, 0x08, 0x2b, 0x06, 0x01, 0x05, 0x05, 0x07, 0x03, 0x01, 0x06, 0x08,
  0x2b, 0x06, 0x01, 0x05, 0x05, 0x07, 0x03, 0x02, 0x30, 0x0d, 0x06, 0x09, 0x2a, 0x86, 0x48, 0x86,
  0xf7, 0x0d, 0x01, 0x01, 0x0b, 0x05, 0x00, 0x03, 0x82, 0x01, 0x01, 0x00, 0xca, 0x49, 0xe5, 0xac,
  0xd7, 0x64, 0x64, 0x77, 0x5b, 0xbe, 0x71, 0xfa, 0xcf, 0xf4, 0x1e, 0x23, 0xc7, 0x9a, 0x69, 0x63,
  0x54, 0x5f, 0xeb, 0x4c, 0xd6, 0x19, 0x28, 0x23, 0x64, 0x66, 0x8e, 0x1c, 0xc7, 0x87, 0x80, 0x64,
  0x5f, 0x04, 0x8b, 0x26, 0xaf, 0x98, 0xdf, 0x0a, 0x70, 0xbc, 0xbc, 0x19, 0x3d, 0xee, 0x7b, 0x33,
  0xa9, 0x7f, 0xbd, 0xf4, 0x05, 0xd4, 0x70, 0xbb, 0x05, 0x26, 0x79, 0xea, 0x9a, 0xc7, 0x98, 0xb9,
  0x07, 0x19, 0x65, 0x34, 0xcc, 0x3c, 0xe9, 0x3f, 0xc5, 0x01, 0xfa, 0x6f, 0x0c, 0x7e, 0xdb, 0x7a,
  0x70, 0x5c, 0x4c, 0xfe, 0x2d, 0x00, 0xf0, 0xca, 0xbe, 0x2d, 0x8e, 0xb4, 0xa8, 0x80, 0xfb, 0x01,
  0x13, 0x88, 0xcb, 0x9c, 0x3f, 0xe5, 0xbb, 0x77, 0xca, 0x3a, 0x67, 0x36, 0xf3, 0xce, 0xd5, 0x27,
  0x02, 0x72, 0x43, 0xa0, 0xbd, 0x6e, 0x02, 0xf1, 0x47, 0x05, 0x71, 0x3e, 0x01, 0x59, 0xe9, 0x11,
  0x9e, 0x1a, 0xf3, 0x84, 0x0f, 0x80, 0xa6, 0xa2, 0x78, 0x35, 0x2f, 0xb6, 0xc7, 0xa2, 0x7f, 0x17,
  0x7c, 0xe1, 0x8b, 0x56, 0xae, 0xee, 0x67, 0x88, 0x51, 0x27, 0x30, 0x60, 0xa5, 0x62, 0x52, 0xc3,
  0x37, 0xd5, 0x3b, 0xea, 0x85, 0x2a, 0x01, 0x38, 0x87, 0xa2, 0xcf, 0x70, 0xad, 0xa4, 0x7a, 0xc9,
  0xc4, 0xe7, 0xca, 0xc5, 0xda, 0xbc, 0x23, 0x32, 0xf2, 0xfe, 0x18, 0xc2, 0x7b, 0xe0, 0xdf, 0x3b,
  0x2f, 0xd4, 0xd0, 0x10, 0xe6, 0x96, 0x4c, 0xfb, 0x44, 0xb7, 0x21, 0x64, 0x0d, 0xb9, 0x00, 0x94,
  0x30, 0x12, 0x26, 0x87, 0x58, 0x98, 0x39, 0x05, 0x38, 0x0f, 0xcc, 0x82, 0x48, 0x0c, 0x0a, 0x47,
  0x66, 0xee, 0xbf, 0xb4, 0x5f, 0xc4, 0xff, 0x70, 0xa8, 0xe1, 0x7f, 0x8b, 0x79, 0x2b, 0xb8, 0x65,
  0x32, 0xa3, 0xb9, 0xb7, 0x31, 0xe9, 0x0a, 0xf5, 0xf6, 0x1f, 0x32, 0xdc
};
const unsigned int caCertLen = 1068;

@tuxedo0801
Copy link
Contributor

How do I extract a cert to create a char caCert[] ??

@madpilot
Copy link

@tuxedo0801 (On *nix) Run it through xxd with the -i flag. So something like xxd -i ca.cert ca_cert.h then open the resultant file, and change ca_cert can ca_cert_len to caCert and caCertLen respectively

@tuxedo0801
Copy link
Contributor

Thanks. And how to I "download" the ca-cert from the target website? I don't have control over the website.

@tuxedo0801
Copy link
Contributor

I followed this howto: http://docs.bvstools.com/home/ssl-documentation/exporting-certificate-authorities-cas-from-a-website

Seems to work. Have to add it to my sketch to check it...

@tuxedo0801
Copy link
Contributor

argh

I'm using Platform.IO with esp core 1.3.1 ... seems that this does not match to 2.4.0-rc1 of esp8266 arduino core...

Question 1) Where do I find the version mapping between the esp8266 arduino core (which is versioned here in this repo) and the correspoding platform.io core package?
Question 2) How do I get the 2.4.0-rc1 version in platform.io, so that I can use/test the ca-cert-tls-stuff?

@tuxedo0801
Copy link
Contributor

Switched back to ArduinoIDE to be able to use 2.4.0-rc1, but sketch crashs now:

Decoding 25 results
0x40223b0e: more_comps at crypto/bigint.c line 672
0x40223ca0: alloc at crypto/bigint.c line 672
0x40223f21: regular_multiply at crypto/bigint.c line 672
0x402249fd: bi_barrett at crypto/bigint.c line 1293
0x40223cbc: alloc at crypto/bigint.c line 672
0x40224bc9: bi_mod_power at crypto/bigint.c line 1400
0x40223b70: trim at crypto/bigint.c line 672
0x40225de5: RSA_public at crypto/rsa.c line 252
:  (inlined by) RSA_encrypt at crypto/rsa.c line 286
0x402217d4: send_client_key_xchg at ssl/tls1_clnt.c line 409
0x40221c69: do_clnt_handshake at ssl/tls1_clnt.c line 123
0x40208458: ClientContext::_consume(unsigned int) at C:\Users\d463\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.0-rc1\libraries\ESP8266WiFi\src/WiFiClientSecure.cpp line 609
:  (inlined by) ClientContext::read(char*, unsigned int) at C:\Users\d463\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.0-rc1\libraries\ESP8266WiFi\src\include/ClientContext.h line 253
0x4020edd8: __yield at C:\Users\d463\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.0-rc1\cores\esp8266/core_esp8266_main.cpp line 96
0x402215dc: do_handshake at ssl/tls1.c line 2005
:  (inlined by) basic_read at ssl/tls1.c line 1481
0x402219b8: do_client_connect at ssl/tls1_clnt.c line 168
0x401004d8: malloc at C:\Users\d463\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.0-rc1\cores\esp8266\umm_malloc/umm_malloc.c line 1664
0x40221744: ssl_read at ssl/tls1.c line 2005
0x402088e1: SSLContext::connect(ClientContext*, char const*, unsigned int) at C:\Users\d463\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.0-rc1\libraries\ESP8266WiFi\src/WiFiClientSecure.cpp line 609
:  (inlined by) WiFiClientSecure::_connectSSL(char const*) at C:\Users\d463\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.0-rc1\libraries\ESP8266WiFi\src/WiFiClientSecure.cpp line 321
0x40207e7a: WiFiClient::connect(IPAddress, unsigned short) at C:\Users\d463\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.0-rc1\libraries\ESP8266WiFi\src/WiFiClient.cpp line 324
0x40208c05: WiFiClientSecure::connect(char const*, unsigned short) at C:\Users\d463\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.0-rc1\libraries\ESP8266WiFi\src/WiFiClientSecure.cpp line 313
0x402038b1: doHttps(String) at C:\dev\MyProject - ArduinoIDE\MyProject/MyProject.ino line 448
0x4020e1f9: String::copy(char const*, unsigned int) at C:\Users\d463\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.0-rc1\cores\esp8266/WString.cpp line 519
0x4020e300: String::operator=(String const&) at C:\Users\d463\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.0-rc1\cores\esp8266/WString.cpp line 519
0x40203d08: loop at C:\dev\MyProject - ArduinoIDE\MyProject/MyProject.ino line 642
0x4020edac: loop_wrapper at C:\Users\d463\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.0-rc1\cores\esp8266/core_esp8266_main.cpp line 124
0x4010070c: cont_norm at C:\Users\d463\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.0-rc1\cores\esp8266/cont.S line 109

What I have done so far:

I extracted the root-ca with help of this howto: http://docs.bvstools.com/home/ssl-documentation/exporting-certificate-authorities-cas-from-a-website
I converted the cert with help of xxd -i ca.cert ca_cert.h to create the arduino code + renamed the variables according to the example code: https://github.com/esp8266/Arduino/blob/master/libraries/ESP8266WiFi/examples/HTTPSRequestCACert/CACert.ino

I copy&pasted the required lines of code from the example sketch: https://github.com/esp8266/Arduino/blob/master/libraries/ESP8266WiFi/examples/HTTPSRequestCACert/HTTPSRequestCACert.ino

and created the following method:

void doHttps(String msg) {
        // Connect to remote server
        Serial.print("connecting to ");
        Serial.println(CLOUD_HOST);
        if (!client.connect(CLOUD_HOST, CLOUD_PORT)) {
                Serial.println("connection failed");
                return;
        }

        // Verify validity of server's certificate
        if (client.verifyCertChain(CLOUD_HOST)) {
                Serial.println("Server certificate verified");
        } else {
                Serial.println("ERROR: certificate verification failed!");
                return;
        }

        Serial.print("requesting URL: ");
        Serial.println(CLOUD_PATH);

        client.print(String("POST ") + CLOUD_PATH + " HTTP/1.1\r\n" +
                     "Host: " + CLOUD_HOST + "\r\n" +
                     "User-Agent: MyProjectESP8266\r\n" +
                     "Connection: close\r\n"+
                     "Accept-Encoding: identity;q=1,chunked;q=0.1,*;q=0\r\n"+
                     "Content-Type: application/json\r\n"+
                     "Authorization: Bearer "+CLOUD_OAUTH_TOKEN+"\r\n"+
                     "Content-Length: "+msg.length()+"\r\n"+
                     "\r\n");
        client.print(msg);

        Serial.println("request sent");
        while (client.connected()) {
                String line = client.readStringUntil('\n');
                if (line == "\r") {
                        Serial.println("headers received");
                        break;
                }
        }
        String line = client.readStringUntil('\n');

        Serial.println("reply was:");
        Serial.println("==========");
        Serial.println(line);
        Serial.println("==========");
        Serial.println();
}

before calling this method, I "initialize" all required stuff for certificate check, like in example sketch (copy&paste):

// Synchronize time useing SNTP. This is necessary to verify that
                        // the TLS certificates offered by the server are currently valid.
                        Serial.print("Setting time using SNTP");
                        configTime(8 * 3600, 0, "pool.ntp.org", "time.nist.gov");
                        time_t now = time(nullptr);
                        while (now < 1000) {
                                delay(500);
                                Serial.print(".");
                                now = time(nullptr);
                        }
                        Serial.println("");
                        struct tm timeinfo;
                        gmtime_r(&now, &timeinfo);
                        Serial.print("Current time: ");
                        Serial.print(asctime(&timeinfo));

                        // Load root certificate in DER format into WiFiClientSecure object
                        bool res = client.setCACert(caCert, caCertLen);
                        if (!res) {
                                Serial.println("Failed to load root CA certificate!");
                                while (true) {
                                        yield();
                                }
                        }

Any ideas what goes wrong?

@tuxedo0801
Copy link
Contributor

ping

@Franciscopcosta
Copy link

Hey @tuxedo0801 did you solved it? I hope so. :)

@tuxedo0801
Copy link
Contributor

No, issue not solved.

For now, I wrote a PHP script that handles incoming HTTP and forwards it to the target HTTPS. But that is not a real and final solution.

@duchere
Copy link

duchere commented Sep 10, 2017

Hi, I have devices in the nature that connect to my server verifying the fingerprint. My issue is that the certificate is expiring in 60 days. Will the library refuse the connection because it's expired or does it only check the fingerprint ?

If it only checks the fingerprint, I assume that my best choice is to keep the expired certificate on my server. I guess I can update OTA my devices every year, with a new firmware that lets them connect to a new server with a non expired certificate then.

Best,

J

@Vladimir-Kondratiev
Copy link

Vladimir-Kondratiev commented Sep 12, 2017

#igrr

Can you add function client.write(webFile) for file upload
It is work with WiFiClient https://github.com/esp8266/Arduino/issues/1853
But not work with WiFiClientSecure
///
error: no matching function for call to 'WiFiClientSecure::write(fs::File&)'
client.write(webFile);
///
#3601

@igrr
Copy link
Member

igrr commented Oct 1, 2017

@tuxedo0801 Could you please open a new issue about the problem you have encountered? Please attach the full .ino file which can be used to reproduce the issue, including the host name you are connecting to, certificates, and so on. Thank you.

@igrr
Copy link
Member

igrr commented Oct 1, 2017

@ilg The issue has been fixed upstream, and the fix was merged in 80e9a84.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests