Add SSL Session capability to speed reconnections

[earlephilhower]

SSL Sessions enable most of the SSL handshake to be skipped when both
client and server agree to use them. Add a BearSSLSession class and
an optional setting to the SSL client to enable this.

Note that SSL sessions are unrelated to HTTP sessions. They are
ephemeral and only relate to the SSL parameters, not anything at
the HTTP protocol level.

Fixes #4796

[earlephilhower]

Here's how much time it saves on an 80MHz core. It's dramatic, when sessions are supported by the server!

Connecting to NOBABIES
...
Connected
IP Address: 
192.168.1.138
Waiting for NTP time sync: .
Current time: Sun Sep 23 22:05:51 2018
Connecting without sessions...Trying: api.github.com:443...Connected!
-------
HTTP/1.1 200 OK
-------

Total time: 2042ms
Connecting with an unitialized session...Trying: api.github.com:443...Connected!
-------
HTTP/1.1 200 OK
-------

Total time: 1564ms
Connecting with the just initialized session...Trying: api.github.com:443...Connected!
-------
HTTP/1.1 200 OK
-------

Total time: 386ms
Connecting again with the initialized session...Trying: api.github.com:443...Connected!
-------
HTTP/1.1 200 OK
-------

Total time: 370ms
Connecting without sessions...Trying: api.github.com:443...Connected!
-------
HTTP/1.1 200 OK
-------

Total time: 1674ms
Connecting with an unitialized session...Trying: api.github.com:443...Connected!
-------
HTTP/1.1 200 OK
-------

Total time: 1597ms
Connecting with the just initialized session...Trying: api.github.com:443...Connected!
-------
HTTP/1.1 200 OK
-------

Total time: 425ms
Connecting again with the initialized session...Trying: api.github.com:443...Connected!
-------
HTTP/1.1 200 OK
-------

Total time: 353ms

[marcelstoer]

Sounds promising! In the issue this fixes it says

it would be great if we can save these params between deepSleep cycles

I don't understand most of the C code but it looks like this isn't (can't be?) supported yet? My real life use-case is exactly that: wake-up, WiFi, NTP, 1 TLS request, processing, deep sleep 30min.

[earlephilhower]

I'm not too familiar with deep sleep modes on the ESP. Do you lose RAM contents? If so, then if you can find enough space for sizeof(BearSSLSession) in persistent memory, it can be saved and reloaded from there.

However, in 30 minutes a heavily used SSL server may not have your session stored anymore. Things will still work, the session will be ignored and you will do a full handshake. If you own the SSL server, you can try and expand the cache and TTL values somehow to help ensure any session token lasts >30 minutes, probably.

[devyte]

@earlephilhower a deepSleep is essentially a reset on the ESP, so yes: you lose everything.
Is the BearSSLSession struct self-contained and contiguous? If so, storing it to flash and reading it back should be easy. If not, serializer/deserializer functions will be needed.
How big is it? Use of a SPIFFS file should be ok in all cases, but if it is small enough, maybe use of the EEPROM class is also possible. Examples should be written as appropriate.

[earlephilhower]

sizeof(BearSSLSession) == 86 and contiguous. All of that is the encryption state, so its size is not negotiable.

Doing a flash overwrite every 30 minutes seems like a dangerous things to do if you want years of lifetime in the field. :(

[devyte]

I was assuming a write only on establishing a new handshake, and no write every time the session is reused. Maybe my thinking is wrong there, and the state changes often?
The worst case would be being forced to do a write every 30mins, which would mean 17520 writes in a year, so yes: EEPROM wouldn't be a good idea for that case. Use of a SPIFFS file would mitigate that due to wear leveling, so that would be much better.

[earlephilhower]

It's mostly identical, but that depends on the server. It is possible to cause SSL renegotiation on a connection at any time, so you may have the same encrypted_state or may not. I suppose a memcmp() would be prudent on the existing stored state before trying to rewrite it. FWIW in the example code the session data is constant for all connections in a series, but that's three connections transferring 10s of bytes in a matter of 1-2 seconds, so not exactly a guarantee.

[artua]

Hi @earlephilhower
Trying to use sessions, it always say "Can't connect". What's wrong?
Using sample code...

Waiting for NTP time sync: ........
Current time: Tue Oct 23 19:12:58 2018
Connecting without sessions...Trying: api.github.com:443...Connected!
-------
HTTP/1.1 200 OK
-------

Total time: 1795ms
Connecting with an unitialized session...Trying: api.github.com:443...*** Can't connect. ***
-------
Total time: 403ms
Connecting with the just initialized session...Trying: api.github.com:443...*** Can't connect. ***
-------
Total time: 407ms
Connecting again with the initialized session...Trying: api.github.com:443...*** Can't connect. ***
-------
Total time: 438ms

[earlephilhower]

Github was having a bad day yesterday, so if this is a log from then I'd just re-try it. If the first "connecting from unitialized session" doesn't work (that's the same as w/o session) then there won't be a session for the second or third to try. It could also be a momentary network hiccup in the 8266.

The other cause might be GitHub changing their SSL certs yet again. I think we've updated hardcoded certs three times this year already.

[artua]

It's from today.
And the first attempt (without sessions enabled) connected without problems.

Connecting without sessions...Trying: api.github.com:443...Connected!
-------
HTTP/1.1 200 OK
-------

[earlephilhower]

I tested it on my system just now and it connected with sessions fine (although it seems the github api returns a "403 Forbidden"...but SSL connects and HTTP is processed to get this error). Are you running a clean git head?

[artua]

I successfully incorporated sessions in my main project.
But with sample project it's still not working.
I even tried client.setInsecure();

// Example of using SSL sessions to speed up SSL connection initiation
//
// September 2018 by Earle F. Philhower, III
// Released to the public domain

#include <ESP8266WiFi.h>
#include <time.h>

const char *ssid="***";
const char *pass="***";

const char *   host = "api.github.com";
const uint16_t port = 443;
const char *   path = "/";

void setup() {
  Serial.begin(115200);
  Serial.println();
  Serial.println();

  Serial.printf("Connecting to %s\n", ssid);
  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, pass);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("\nConnected");
  Serial.println("IP Address: ");
  Serial.println(WiFi.localIP());

  // Set up time to allow for certificate validation
  configTime(3 * 3600, 0, "pool.ntp.org", "time.nist.gov");

  Serial.print("Waiting for NTP time sync: ");
  time_t now = time(nullptr);
  while (now < 8 * 3600 * 2) {
    delay(500);
    Serial.print(".");
    now = time(nullptr);
  }
  Serial.println("");
  struct tm timeinfo;
  gmtime_r(&now, &timeinfo);
  Serial.print("Current time: ");
  Serial.print(asctime(&timeinfo));
}

// Try and connect using a WiFiClientBearSSL to specified host:port and dump HTTP response
void fetchURL(BearSSL::WiFiClientSecure *client, const char *host, const uint16_t port, const char *path) {
  if (!path) {
    path = "/";
  }

  Serial.printf("Trying: %s:443...", host);
  client->connect(host, port);
  if (!client->connected()) {
    Serial.printf("*** Can't connect. ***\n-------\n");
    return;
  }
  Serial.printf("Connected!\n-------\n");
  client->write("GET ");
  client->write(path);
  client->write(" HTTP/1.0\r\nHost: ");
  client->write(host);
  client->write("\r\nUser-Agent: ESP8266\r\n");
  client->write("\r\n");
  uint32_t to = millis() + 5000;
  if (client->connected()) {
    do {
      char tmp[32];
      memset(tmp, 0, 32);
      int rlen = client->read((uint8_t*)tmp, sizeof(tmp) - 1);
      yield();
      if (rlen < 0) {
        break;
      }
      // Only print out first line up to \r, then abort connection
      char *nl = strchr(tmp, '\r');
      if (nl) {
        *nl = 0;
        Serial.print(tmp);
        break;
      }
      Serial.print(tmp);
    } while (millis() < to);
  }
  client->stop();
  Serial.printf("\n-------\n\n");
}


void loop() {
  static const char digicert[] PROGMEM = R"EOF(
-----BEGIN CERTIFICATE-----
MIIDxTCCAq2gAwIBAgIQAqxcJmoLQJuPC3nyrkYldzANBgkqhkiG9w0BAQUFADBs
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3
d3cuZGlnaWNlcnQuY29tMSswKQYDVQQDEyJEaWdpQ2VydCBIaWdoIEFzc3VyYW5j
ZSBFViBSb290IENBMB4XDTA2MTExMDAwMDAwMFoXDTMxMTExMDAwMDAwMFowbDEL
MAkGA1UEBhMCVVMxFTATBgNVBAoTDERpZ2lDZXJ0IEluYzEZMBcGA1UECxMQd3d3
LmRpZ2ljZXJ0LmNvbTErMCkGA1UEAxMiRGlnaUNlcnQgSGlnaCBBc3N1cmFuY2Ug
RVYgUm9vdCBDQTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMbM5XPm
+9S75S0tMqbf5YE/yc0lSbZxKsPVlDRnogocsF9ppkCxxLeyj9CYpKlBWTrT3JTW
PNt0OKRKzE0lgvdKpVMSOO7zSW1xkX5jtqumX8OkhPhPYlG++MXs2ziS4wblCJEM
xChBVfvLWokVfnHoNb9Ncgk9vjo4UFt3MRuNs8ckRZqnrG0AFFoEt7oT61EKmEFB
Ik5lYYeBQVCmeVyJ3hlKV9Uu5l0cUyx+mM0aBhakaHPQNAQTXKFx01p8VdteZOE3
hzBWBOURtCmAEvF5OYiiAhF8J2a3iLd48soKqDirCmTCv2ZdlYTBoSUeh10aUAsg
EsxBu24LUTi4S8sCAwEAAaNjMGEwDgYDVR0PAQH/BAQDAgGGMA8GA1UdEwEB/wQF
MAMBAf8wHQYDVR0OBBYEFLE+w2kD+L9HAdSYJhoIAu9jZCvDMB8GA1UdIwQYMBaA
FLE+w2kD+L9HAdSYJhoIAu9jZCvDMA0GCSqGSIb3DQEBBQUAA4IBAQAcGgaX3Nec
nzyIZgYIVyHbIUf4KmeqvxgydkAQV8GK83rZEWWONfqe/EW1ntlMMUu4kehDLI6z
eM7b41N5cdblIZQB2lWHmiRk9opmzN6cN82oNLFpmyPInngiK3BD41VHMWEZ71jF
hS9OMPagMRYjyOfiZRYzy78aG6A9+MpeizGLYAiJLQwGXFK3xPkKmNEVX58Svnw2
Yzi9RKR/5CYrCsSXaQ3pjOLAEFe4yHYSkVXySGnYvCoCWw9E1CAx2/S6cCZdkGCe
vEsXCS+0yx5DaMkHJ8HSXPfqIbloEpw8nL+e/IBcm2PN7EeqJSdnoDfzAIJ9VNep
+OkuE6N36B9K
-----END CERTIFICATE-----
)EOF";
  uint32_t start, finish;
  BearSSL::WiFiClientSecure client;
//  client.setInsecure();
  BearSSLX509List cert(digicert);
  client.setTrustAnchors(&cert);

  Serial.printf("Connecting without sessions...");
  start = millis();
  fetchURL(&client, host, port, path);
  finish = millis();
  Serial.printf("Total time: %dms\n", finish - start);

  BearSSLSession session;
  client.setSession(&session);
  Serial.printf("Connecting with an unitialized session...");
  start = millis();
  fetchURL(&client, host, port, path);
  finish = millis();
  Serial.printf("Total time: %dms\n", finish - start);

  Serial.printf("Connecting with the just initialized session...");
  start = millis();
  fetchURL(&client, host, port, path);
  finish = millis();
  Serial.printf("Total time: %dms\n", finish - start);

  Serial.printf("Connecting again with the initialized session...");
  start = millis();
  fetchURL(&client, host, port, path);
  finish = millis();
  Serial.printf("Total time: %dms\n", finish - start);

  delay(10000); // Avoid DDOSing github
}