Add rate limiting to the XMLRPC Api

[dstufft]

Currently whenever we get a big burst of XMLRPC requests, our service can start timing out and raising 5xx errors. We should introduce rate limiting in the XMLRPC API in order to force clients to slow their roll and not hammer us too hard.

[di]

Added in #4553.

[pradyunsg]

Can we increase this limit a little bit?

pip's test suite seems to be hitting this limit in CI, which has broken all our CI pipelines and is blocking all PR merges. :(

eg:

• https://dev.azure.com/pypa/pip/_build/results?buildId=29507&view=logs&j=88d9dc86-3d79-5e8a-7a2d-b92d46a16d40&t=6bf4b43e-ad63-576f-33cc-d8124dfcf590&l=258
• https://dev.azure.com/pypa/pip/_build/results?buildId=29507&view=logs&j=f801bdd4-2ff4-5da8-3bd6-deea013b4470&t=ada738ba-3760-52e8-582e-eef0693a80cb&l=260

/cc @ewdurbin @brainwane

[di]

@pradyunsg We recently reduced the limit because the API was being abused and exhausting PyPI's backend resources.

The current limit is 2 per second / 30 per minute, so depending on how pip uses the XML-RPC APIs, you could add a delay to the affected tests.

[pradyunsg]

Coolieo. pypa/pip#9051 should make it possible to work around those.

FWIW, I'm a +1 on giving a farewell to the search endpoint.