-
Notifications
You must be signed in to change notification settings - Fork 217
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
manylinux1 packages contain unstripped, but unstrippable, shared libraries #78
Comments
I suspect that the reason they can't be stripped is a side effect of the way auditwheel uses patchelf to patch binaries when it's bundling libraries. I think the solution is probably for auditwheel to be taught how to automatically/optionally strip binaries. |
Do you know how much stripping actually affects these binaries? |
i wondered the same question, "how much is stripping these binaries saving", but realized after a second or two that without being able to strip the binaries (the very thing that is failing and i tried on ubuntu 14.04 and 16.04 with no success) that i'm not sure of an easy method to calculate the size reduction (as i don't look forward to manually reviewing the symbols in the shared libraries, resolving which are not necessary, and calculating their collective size; that's what strip is for 😉). and it's not so much that these specific libraries can't be stripped, regardless of their size as they are just prominent examples, but that stripping fails on manylinux1 packages, which is quite the oddity/irregularity/abnormality. i can alter my predeployment-packaging step to ignore this particular failure while stripping shared libraries, but figured i would report it "upstream" first because it is definitely "unique". thanks for the tip; i'll look into auditwheel (if only out of curiosity and enlightenment). |
okay, after reading through auditwheel, specifically repair.py and noting it's strong dependency on patchelf (as you mentioned, @njsmith), i found this similar-sounding patchelf issue: NixOS/patchelf#10. |
I did some stripping tests on numpy / scipy / scikit-learn in the past (before manylinux existed) and the size reduction can be very large: a typical scipy stack venv would go from ~250MB down to less than 50MB. This was for instance documented in: https://serverlesscode.com/post/deploy-scikitlearn-on-lamba/ |
I agree if the problem cannot be easily fixed at the patchelf level. |
It would be very difficult to fix this at the patchelf level.
…On Jan 7, 2017 3:58 AM, "Olivier Grisel" ***@***.***> wrote:
@njsmith <https://github.com/njsmith> I think the solution is probably
for auditwheel to be taught how to automatically/optionally strip binaries.
I agree if the problem cannot be easily fixed at the patchelf level.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#78 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAlOaML5wiQoqvWGiWFEWQl4gTf_hAO6ks5rP33hgaJpZM4JUsG5>
.
|
@njsmith I tried to build this suggested fix from NixOS/patchelf#10 (comment) but I cannot get the bootstrap.sh script to work on our manylinux1 docker container. How did you generate the distribution files in the https://nipy.bic.berkeley.edu/manylinux/patchelf-0.9njs2.tar.gz tarball? |
Olivier - I think that's me. I just ran the bootstrap.sh script on another machine with more recent autoconf tools, and then packed up the archive with the resulting configure file. Is that what you mean? |
Yes I tried something like that but also need other files no? Like build-aux or something. |
I just checked out that commit, and ran |
I tried again and it worked. I don't understand what I did wrong the first time. Sorry for the noise. |
NixOS/patchelf#117 has been merged - it'd be great if the Docker images could include this fix soon, since this breaks the Debian virtualenv packaging workflow: spotify/dh-virtualenv#150 |
@sciyoshi: any interest in working on a patch? |
Update patchelf from latest master to fix #78
There are manylinux1 packages on PyPI that contain unstripped shared libraries that cause
strip
to fail.This behavior has been observed on both Debian 8, Ubuntu 14.04, and Ubuntu 16.04.
If this is not a general artifact of the manylinux build process (and therefor not applicable to this project specifically), but a common (mis)configuration that affects multiple packages' manylinux1 build configuration/settings (eg cffi, numpy), then please let me know and I'll close this issue (and reopen it against the relevant projects).
The text was updated successfully, but these errors were encountered: