-
-
Notifications
You must be signed in to change notification settings - Fork 29.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
venv module does not copy the correct python exe #87915
Comments
On windows, the venv module does not copy the correct python exe if the current running exe (eg sys.executable) has been renamed (eg, named python3.exe) venv will only make copies of python.exe, pythonw.exe, python_d.exe or pythonw_d.exe. If for example the python executable has been renamed from python.exe to python3.exe (eg, to co-exist in a system where multiple pythons are on PATH) then this can fail with errors like: Error: [WinError 2] The system cannot find the file specified When venv tries to run pip in the new environment. If the running python executable is a differently named copy then errors like the one described in https://bugs.python.org/issue40588 are seen. |
This may also cause https://bugs.python.org/issue35644 |
I just encountered the same behavior. Steps to reproduce:
The environment is created up until the point _setup_pip() is called in venv, which causes the following error to be printed:
Running venv from the interpreter gives the full stack trace. $ python3 -c 'import venv; venv.main([".venv"])'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\niklas\AppData\Local\Programs\Python\Python39\lib\venv\__init__.py", line 491, in main
builder.create(d)
File "C:\Users\niklas\AppData\Local\Programs\Python\Python39\lib\venv\__init__.py", line 75, in create
self._setup_pip(context)
File "C:\Users\niklas\AppData\Local\Programs\Python\Python39\lib\venv\__init__.py", line 299, in _setup_pip
subprocess.check_output(cmd, stderr=subprocess.STDOUT)
File "C:\Users\niklas\AppData\Local\Programs\Python\Python39\lib\subprocess.py", line 424, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "C:\Users\niklas\AppData\Local\Programs\Python\Python39\lib\subprocess.py", line 505, in run
with Popen(*popenargs, **kwargs) as process:
File "C:\Users\niklas\AppData\Local\Programs\Python\Python39\lib\subprocess.py", line 951, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "C:\Users\niklas\AppData\Local\Programs\Python\Python39\lib\subprocess.py", line 1420, in _execute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] The system cannot find the file specified |
To complete my previous comment, patching the function to see the command that is invoked, you can see it tries to invoke 'python3.exe' (seemingly derived from sys.executable) in the venv but the venv only contains 'python.exe'. ['C:\\Users\\niklas\\dotfiles\\.venv\\Scripts\\python3.exe', '-Im', 'ensurepip', '--upgrade', '--default-pip'] |
Hey, can I request a backport of this change? For my use case, 3.9+ would be great, but going back to 3.7 should be viable? I trust your judgement :) I'll happily create the PRs for the backport, but it looks like the bug has to be indicated as "needs backport to X.Y" by Python maintainers first (https://devguide.python.org/gitbootcamp/?highlight=backport#backporting-merged-changes) |
3.10 and 3.9 - OK, but 3. is security fixes only, I'm afraid. |
*3.8, I meant. |
That's great, thanks! |
The Windows implementation of symlink_or_copy() actually copies "python.exe" and "pythonw.exe" launchers from "Lib/venv/scripts/nt". One cannot simply copy the "python3.exe" executable because the required DLLs aren't copied. If this seemed to work when testing, it was only because the installation directory was in PATH. The solution that actually works is to copy the launcher as "python3.exe". |
We'd also need to update the launcher to launch the executable with its name, which it currently doesn't do. I was looking at this recently for some other reason. |
The code to copy a file could be rewritten to use a regex match. For example:
|
Everywhere: on CI, in tests, in docs. Also, we replace bare pip/pip3 and pytest calls with python -m pip and python -m pytest respectively, to make sure we always use the same python executable everywhere. This fixes #326 (failing Windows tests on CI), because the `python3` executable causes issues (it is not properly copied into the venv directory by venv, which is a CPython bug, see python/cpython#87915), while the python executable does work; see e.g. here https://stackoverflow.com/questions/61669873/python-venv-env-fails-winerror-2-the-system-cannot-find-the-file-specified. `python` should be the same executable under the hood as `python3`, if the setup-python action works as expected, at least. It also allows us to remove the IS_WINDOWS_CI special case. Failed attempts at fixing #326 included: - Setting the shells (assuming something went wrong with environment variables) - Using full paths for both the executable and the venv directory in the template test suite.
Everywhere: on CI, in tests, in docs. Also, we replace bare pip/pip3 and pytest calls with python -m pip and python -m pytest respectively, to make sure we always use the same python executable everywhere. This fixes #326 (failing Windows tests on CI), because the `python3` executable causes issues (it is not properly copied into the venv directory by venv, which is a CPython bug, see python/cpython#87915), while the python executable does work; see e.g. here https://stackoverflow.com/questions/61669873/python-venv-env-fails-winerror-2-the-system-cannot-find-the-file-specified. `python` should be the same executable under the hood as `python3`, if the setup-python action works as expected, at least. It also allows us to remove the IS_WINDOWS_CI special case. Failed attempts at fixing #326 included: - Setting the shells (assuming something went wrong with environment variables) - Using full paths for both the executable and the venv directory in the template test suite.
hi, this problem is still existed in recently, how to solve it? |
@lmy668 what version do you use? |
@xingdongzhe i tried to python3.8.10 and 3.10.5, it's both not . |
@lmy668 I do these on my windows , it works fine. Hope that help you too python3.8.10 -m venv venv # this will error with [WinError 2], but has venv dir
cp venv/Scripts/python.exe venv/Scripts/python3.8.10
python3.8.10 -m venv venv |
I would suggest at least adding a But also, Python 3.8 is not getting bugfixes anymore unless there's a security aspect. You won't see any fixes there. 3.10.5 ought to have the first fix, but I don't recall what else needed to be done. In any case, demonstrating a problem in anything besides 3.11, 3.12 or 3.13 is not going to help move this issue forward. |
any reply is appreciating. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: