pip-20.3b1 interprets httpie 'requests[socks]>=2.22.0' requirement differently than "legacy-resolver" #9112
Labels
type: support
User Support
Comments
|
more visual too_long result: |
|
the pip-20.2.4 output for comparaison: |
|
hum, I notice that the "--pre" option seems to have not work ?? pip-20.3.0b1 output instead of pip-20.2.4 output |
|
pip is switching to a much stricter dependency resolver in 20.3 that checks all the dependencies are satisfied. A 2x slow down is not unreasonable in such a large requirement set with open version specifiers depending on the same packages. The slowdown may happen even when the output is seemingly identical, since the new resolver needs to check things, while the old resolver just happily goes through the list once and calls it a day. You’ll need to profile the run if you suspect there is a bug causing the slowdown. As for |
|
hi, just trying to confirm the "--pre" was in my test. |
|
so doing it the "build script" way, the "--pre" problem seems to go away, so that may have been my fat fingers. ... still seems going throught same slowness. |
|
I don't see anything speciale in slicer "setup.py"... no apparent dependancy at all, except the classifiers; Could it be that your are picky on classifier (no Python-3.9) ? I would suspect this may be the problem I notice the METADATA of slicer includes documentation, maybe something not normal ? |
|
.. it spins forever. so the "--pre" works but is not the problem... no clue why it seems to go in spiral, except the classifier. |
|
seems to not go through either with Python-3.8... will give it 1 hour chance. I don't even guess why it takes requests-2.23.0 (at least at first) when requests-2.24.0 was available and AFAIK OK. |
|
@stonebig try with the current master branch instead? There's a couple of minor bug fixes here. |
|
so, to check on previous run with pip-20.2.4, I don't think there is any packages requesting requests-2.23.0 over requests-2.24.0 |
|
@pradyunsg there is only one fix added. #9085 . you think it's related to my problem ? |
|
giving master a chance.. |
|
doesn't seem to change anything on the request-0.23.0 marker |
|
Damm even I am having a similar kind of issue, I have searched all over the internet and even have posted on number of threads on different forum, no solution seems to work. I am really frustrated, can anyone of you here help me resolve this issue, I am very much tired now. |
|
@deolsunny533 if your example is smaller than mine , maybe shall you give it to help inquiry |
|
if this can help, I retried my example today with latest "ordered" master. it seems the wheel requests_toolbelt-0.9.1-py2.py3-none-any.whl makes that bug triggered ? pipdeptree says: instruction: result_2020-11-11_master.txt (breaking when it start to take forever and my resuests-0.23.0 appears: |
|
this requirement doesn't trigger the bug... this one neither: |
|
this subset does trigger the bug this smaller one does trigger the bug: this one doesn't trigger the bug.... this smaller does trigger the bug this smaller one triggers the bug this one doesn't trigger the bug, i just removed httpie.... this smaller on triggers my supected bug: this smaller one triggers my problem this smaller one triggers my problem |
Likely one of the packages has a version that rejects 0.24.0, and the resolver therefore chose 0.23.0. That version rejecting requests 0.24.0 was not eventually chosen (for some other reason), but the previous decision to choose requests 0.23.0 is kept since it is “good enough” (because the user and all chosen packages say it’s OK). It is easy to tell in hindsight that the resolver could’ve chosen 0.24.0, but the resolver did not know if that additional trail would be a significant time sink when the decision needs to be made. If this is a problem for you, you should specifically ask for 0.24.0. If you don’t do that, the resolver is allowed to choose 0.23.0, even if it is not the absolutely best result. |
|
normally there is no rejection directive on requests , there is only ">=" limitations, except moviepy==1.0.3 [requires: requests>=2.8.1,<3.0]. |
|
hum looking at httpie setup.py there is this subtle thing: |
|
The only way to definitely know what happened to requests 2.24.0 is to manually retrace every step the resolver did. I’m definitely not going to do that, and would suggest others not to as well, since I implemented the resolver exactly to avoid having to worry about this kind of details. I guess there’s no harm spending time persuading it for intellectual interest though, if you find it worthwhile. |
|
so this simple requirement file makes me go down on master (versus current pip that doesn't complain). even this single package one further simplifying still the issue not using requirement file |
|
The issue goes away if I include the wheel "PySocks-1.7.1-py3-none-any.whl" in my wheel directory "C:\WinP\packages.srcreq".. so:
who has a functionnal bug:
|
|
so "httpie" package requiring apparently "socks" package, because of 'requests[socks]>=2.22.0', was my only problem |
stonebig
changed the title
stonebig
changed the title
|
hum, on pip-20.3 master, "pip show" is not conform to "pip install" for httpie: it doesn't specify socks. maybe it shall be considered a bug |
stonebig
changed the title
|
AFAIK, httpie does specify Am I correct to assess the situation as the new resolver doing the right thing and detects a user error that previously went unoticed? |
|
When we specify dask[bag] , we don't ask for a "bag" package, so I'm a little confused about what is correct or not. And requests doesn't seem to have an extra [socks] option, so I truly don't understand the mechanism on this case, old an new method. |
|
with latest proposed pached this morning here, my bug (missing pysocks wheel in my private wheel directory) gets now a resolution... I don't know if it's normal either, it's slow, but it comes to an end which is goooood. giving |
|
reading further discussions, I discover the existence of the contraints.txt option , that rather solves my problems of resolution time (when there is in fact no solution) with the new-resolver. |
|
testing with today #9264, my problem is still resolved, still slow but still good. |
|
by the way, I still don't have a nice way to explain me the conflicts (here a missing wheel) that launches the backtracking:
a missing wheel (no wheel at all) in my local directory of wheels is my principal issue, in my process. |
|
Are all the |
|
Here the.only problem is there is zero pysocks wheel in my dedicated directory. It took me a looong time to figure it out, there was no clear reference to that problem in the resolver, --verbose doesn't seem more verbal about this, and pip check doesn't speak about it... when you backpedal on a given wheel, maybe shall you state the 1st constraint that you try to remove, give a clue. Typically, I would like to see "socks needed and missing" , at least once |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
This is by no mean an exact timing test. I hope I kept enough exact comparison environment.
pip-20.3.0b1 takes at least (if it goes till the end) twice more time on below example.
Is it supposed to work well with "--pre --no-index --trusted-host=None --find-links=a_local_directory" ?
Is the option "--pre" still valid ?
I'm preparing Winptyhon builds using a local wheel directory "C:\WinP\packages.srcreq"
I install the attached requirement "Cod_requirements64.txt" on a Python-3.9.0.1 64 bit Windows using:
intruction is
pip install --pre --no-index --trusted-host=None --find-links=C:\WinP\packages.srcreq -r requirements.txtResult with pip-20.2.4:
Result with pip-20.3.b1:
Cod_requirements64.txt
pip-20.3.0b1_too_long_test_output.txt
pip_20.2.4_installed_package_result.txt
Addendum of november 11th: after resolution of the problem, speed of Pip-20.3 master is roughly 15' to install WinPython packages, pretty much equal to previous resolver.
The text was updated successfully, but these errors were encountered: