Fix Stripe Client Telemetry

[jameshageman-stripe]

In #518 I erroneously stored request metrics on the APIRequestor, as a new instance of APIRequestor is created on each request. This PR moves the tracking of request metrics to the base HTTPClient, so that metrics can be stored between requests.

This PR depended on #526, which changed the client to reuse the HTTPClient between requests in the default case.

r? @ob-stripe @brandur-stripe
cc @dcarney-stripe @akropp-stripe

[ob-stripe]

@jameshageman-stripe You can rebase on the latest master to fix the annoying PyPy3 build issue.

Overall this looks good to me, minus the potential thread-safety issue I mentioned in #526 (comment). I think API requests themselves might still work (I just did a simple test and nothing visibly blew up) but threads would probably overwrite each other's metrics if the client instance is shared.

[jameshageman-stripe]

@ob-stripe We might be fine by just storing _last_request_metrics on thread local storage if the rest of the client code is thread safe (but I don't know for sure if it is).

[jameshageman-stripe]

@ob-stripe @brandur-stripe I've updated the client to store thread local metrics.

I stress tested the client (on master) with multithreading and the only thing I saw was the occasional network error (broken pipe or connection reset by peer). These were unpredictable, and could be avoided by setting stripe.max_network_retries = 3.

I don't know how we'd create a thread local http client when the client needs to live on stripe.default_http_client, unless we set that to some sort of proxy object that delegates to thread local storage. None of that would work anyway if the user explicitly set stripe.default_http_client, so I think we are stuck supporting clients being shared between threads.

[brandur-stripe]

Thanks for running those extra threading tests James!

This LGTM, but going to leave final okay to OB lest I make another merge faux pas :) cc @ob-stripe (And BTW, may want to "squash and merge" this one unless James rebases.)

[jameshageman-stripe]

@brandur After rebasing over #531 , I'm seeing this exception in my integration test:

Traceback (most recent call last):
  File "/home/travis/build/stripe/stripe-python/stripe/http_client.py", line 241, in request
    result = self._thread_local.session.request(
AttributeError: '_thread._local' object has no attribute 'session'

Which I find puzzling since self._thread_local.session is set in the RequestsClient constructor, which is being called before any requests are made. I'll investigate some more.

@ob-stripe aside from my one failing test, could you review?

[jameshageman-stripe]

The broken test should be fixed after #534 is merged.

[brandur-stripe]

Looks like those new integration tests might really help suss out some bugs in the future :)

But yep, sorry about the breakage here. We should have #530 in soon.

[brandur-stripe]

Left a couple more minor comments here, but I think we're close to ready to go.

ptal @jameshageman-stripe

[brandur-stripe]

Just now that I'm taking a second look at this, would you mind moving this into a helper method called something like _add_telemetry_header(headers)? Just so we have less code in the main body here.

[brandur-stripe]

And maybe extract this one to something like _store_telemetry(request_start, rheaders).

You may want to unwind _last_request_metrics and _set_last_request_metrics into the new helpers too.

[jameshageman-stripe]

Done!
ptal @brandur-stripe

[brandur-stripe]

Thanks @jameshageman-stripe!

@ob-stripe Going to pull this one in for now, but if you'd like any additional changes, let us know.

[brandur-stripe]

Released as 2.20.3.