Drop Support for Python 2 in pip 21.0

[dstufft]

Placeholder issue for dropping support for Python 2. Discussion is largely occurring at https://discuss.python.org/t/packaging-and-python-2/662.

Related: #4350.

[pradyunsg]

We've documented how we're gonna handle Python 2 support going forward, at https://pip.pypa.io/en/latest/development/release-process/#python-2-support.

Python 2 support

pip will continue to ensure that it runs on Python 2.7 after the CPython 2.7 EOL date. Support for Python 2.7 will be dropped, if bugs in Python 2.7 itself make this necessary (which is unlikely) or Python 2 usage reduces to a level where pip maintainers feel it is OK to drop support. The same approach is used to determine when to drop support for other Python versions.

However, bugs reported with pip which only occur on Python 2.7 would likely not be addressed directly by pip’s maintainers. Pull Requests to fix Python 2.7 only bugs will be considered, and merged (subject to normal review processes). Note that there may be delays due to the lack of developer resources for reviewing such pull requests.

[pfmoore]

It might be worth considering what we'd do if one of our vendored dependencies dropped support for Python 2. We would basically have two options - freeze the version of the dependency that we vendored, or drop support for Python 2 ourselves. Neither seems ideal, but I assume we'd probably have to freeze the version unless/until the changes in the vendored library became too significant enough for that to be viable (e.g., a security issue).

I'd be a strong -1 on vendoring two versions, one that supports Python 2 and one that doesn't - that would be unreasonable bloat as well as a maintenance overhead. But I guess another option might be to start shipping separate Python 2 and Python 3 versions of pip in that case (but again, that's probably more of a maintenance overhead than I'd be OK with).

I've no idea what our dependncies' plans are, but having a documented plan of action on how we'll react is probably good.

My preferred option would be to say that we'd stick on the last version that supported Python 2 for minor dependencies but if a major dependency dropped Python 2 support, we'd expect to follow suit. What's "minor" would be a judgement call, but I'd probably class things like requests, html5lib and CacheControl as major (mainly because of the risk of not getting security fixes).

[pradyunsg]

I recently started a pip-maintainers email thread (basically, an email with To: containing everyone with write access to this repo) to talk about pip's Python 2 support plans, with 2 specific questions:

1. What metric do we want to use?
2. What %-age of that metric do we stop supporting Python 2 support?

Our answers ended up along the lines of:

1. PyPI file downloads/simple downloads w/ Python 2.7 using the latest pip (that's been out for more than a month), are the best metrics we have.
2. ~20%, with 3 releases that print "okay, Python 2.7 support in pip is gonna go away in release YY.N".

---

I looked at our metrics for the last few weeks and... we've hit the "~20%" target in April 2020! So, pip 20.1 is going to be the first version to start printing a warning containing the exact version of pip that will drop Python 2 support. pip 20.3 will be the last version of pip that supports Python 2.7.

---

Picking out some of the statements from that thread (without attribution):

I don’t care about the specific numbers at play, 20%, 15%, 10%, whatever they’re all roughly the same to me. I would even be OK with calling the number 20%, and starting the process now at this point. My only real caveat to that is I think that the higher the target number is, the more releases we should give a warning in before we drop it.

I think that the main thing we should do is give a very clear indication of the timeline, and what people need to do. [snip] we should start sooner rather than later emphasising the point that this does not mean that "Python 2 users will no longer be able to use pip" but rather that "Python 2 users will have to pin to a specific pip version".

I was recently thinking about suggesting dropping Python 2 support in 21.0 (1 year after CPython) [snip] So +1 from me also. A deprecation could be added in 20.1

Another reason I like this target, is that Python 3.5 goes EoL in 2020-09, which means pip 20.3 should be the last version to support Python 2.7 and/or Python 3.5. We'll be able to use f-strings in pip's codebase starting October this year!

[pradyunsg]

pip 20.1 is going to be the first version to start printing a warning containing the exact version of pip that will drop Python 2 support.

#8087 does this.

[gaborbernat]

I don't view this a binary state necessarily. It's fine to not support new features post drop, but any plans with bug fix releases past that point (this is hard with pips calver to be fair). But assume pips new resolver goes into infinite loop for some reason on python 2 we should offer some bug fix patch release from community, rather than abandon them entirely 🤷‍♂️

[pradyunsg]

It's fine to not support new features post drop

That's precisely the state pip is in right now. I think we have code in pip master currently that conditionally gives a better experience on Python 3, but doesn't on Python 2.

assume pips new resolver goes into infinite loop for some reason on python 2 we should offer some bug fix patch release from community, rather than abandon them entirely 🤷‍♂️

If there's a show-stopper bug like that, I'm sure we'll know about it during the 3 months between 20.3 and 21.0, and we'll make a bugfix then. I don't think it's a good use of our limited developer time to spend any effort on Python 2 after 21.0 goes out -- Linux distributions and other redistributors that promised longer support cycles can pick up that workload, once upstream pip drops support (FWIW, Python 2 users likely already do this for Python itself).

I'll flag that existing versions of pip will continue working on Python 2 and given that Python 2 is not a moving target anymore, that should continue to be the case.

[ThiefMaster]

but rather that "Python 2 users will have to pin to a specific pip version".

I guess you'll use python_requires so pip install -U pip will still be fine and not suddenly install an incompatible pip version?

[pradyunsg]

@ThiefMaster Yup.

[dstufft]

I'd also point out that we're not legally obligated never to release another version that supports 2.x ever again. If we get half way through 2021 and some horrible show stopping bug that only affects 2.x (or more specific, <21) emerges, we can just release a bugfix for 20.3. The third digit in our version number is not calver based specifically for this reason. We can release 20.3.1 (or whatever) in 2021 if needed.

Of course by dropping support we're explicitly saying that we have no plans to do that AND we make it more difficult to do so because the delta between master and 20.3 will continue to grow. We can "roll with the punches" as needed going into the future, with the expectation being that unless something extraordinary happens, there will be no 2.x supporting release in 2021+.

[pradyunsg]

Alrighty! Merged the PR (#8087) that'll make pip 20.1 mention that pip 21.0 is the release that'll drop support for Python 2, when pip is run on Python 2.