GHC too big for Hydra (aarch64 ATM) #66277
Comments
vcunat
added
0.kind: bug
6.topic: haskell
0.kind: regression
|
Recent successful build has output size 2040 MiB, so I guess 2 GiB is the limit, whereas x86_64-linux is still below 1.7 🤷♀️ |
Needed temporarily because of NixOS#66277.
Column #1 is (size on aarch64 - size on x86) in bytessame list, sorted by diff: |
|
I suspect this might become a notable issue for 19.09. /cc release managers @disassembler, @lheckemann. |
|
While it's sad, I'm not sure I can do much about it, lacking ghc knowledge. Maybe Someone™ can do more, but there hasn't been much response to @grahamc's ghc issue yet (and I think that would be the best place, unless we opt to increase the maximum?). |
|
I mainly meant it like... you should know about it. AFAIK one way also might be to default to some older GHC version for aarch64, for now. |
|
8.4.4 has been succeeding (except for occasional time-outs)... so any arguments against defaulting aarch64 to that version? For now at least, but I can't see this moving forward so it might well remain that way for 19.09. |
|
I should've pinged ghc maintainers: @MarcWeber, @kosmikus, and @peti (who still seems active around here). |
|
The only issue with that is it kind of breaks expectations to have a different default depending on the platform. Were I to decide, I would pin the same version by default on all platforms. Though, I do not know what issues this could bring. |
|
I believe it's possible to disable the build of ghc's documentation for aarch64. Maybe that would help? |
|
In that case, would it be better to get the docs in a separate output? (Or is the maximum size all outputs combined?) |
|
It is a separate output now, I believe. (It has "doc" output, at least.) |
|
I'm sure y'all know this, but: pulling the docs out doesn't solve the problem that individual |
|
Last successful 8.6.5 had docs with NarSize 252 MiB, so that wouldn't be enough, according to Graham:
I expect the real issue isn't nix-specific and belongs better upstream, but in the meantime I'd expect we provide at least some binaries. Besides the older ghc version, bypassing the limit might be a nice way (perhaps temporarily and specifically for this case), but I have no idea how to do that. |
|
My understanding is that llvm version might make a significant difference, assuming multiple versions even work. |
|
Experiment 1: default ghc with llvm 7 It complains, but it succeeds and the resulting ghci starts for me. That seems usable. Any thoughts? |
|
Experiment 2: with llvm 5, basically the same |
|
Oh, wait... it's the same size with the default (llvm 6) ... then I don't understand Graham's comment about the size being about half a GB larger 🤷♀️ |
|
Removing most docs in 4179ba0 lead to success. Any comments? For example, is it easy to avoid building docs instead? I plan to apply this work-around on other affected versions as well (as I hate these wasted expensive builds), though perhaps not by copy&paste but something like |
Could not be easier. :-) This line is your friend: If that conditional is It might be worthwhile passing EDIT: Um, it's probably obvious in any case, but just to be sure ... the conditional for cross-compilation does more than just disable the documentation, of course. The part of that snippet that's relevant for our purposes is this one: nixpkgs/pkgs/development/compilers/ghc/8.6.5.nix Lines 65 to 67 in 4179ba0 |
|
We're getting worse: binary 8.6.5 is >2G even without docs, so we can't bootstrap the newly-default 8.8.3. Our compiled ghc865 builds fine, still with the hack that deletes docs to fit under 2G. EDIT: the compiled 8.8.3 is even bigger: |
|
@bgamari I'm looking at the official 8.10 ARM bindist https://downloads.haskell.org/~ghc/8.10.1/ghc-8.10.1-aarch64-deb9-linux.tar.xz (hopefully that is representative for what nix builds here) and wonder: Each Also, what can possibly be in that thing that it's 230 MB in size? |
Right. If it compresses well, we should make use of that in my opinion -- often the cost of engineering time spent at dealing with more sophisticated solutions easily outweighs the saving gained by coming up with them. |
|
It takes 2 real gb of space on users' disks. |
|
or 4, or 6, or 8, or more, since people who have ghc at all are likely to have multiple ghcs in their nix store :) |
|
An example recurring issue is |
|
|
|
We can't patchelf+strip due to patchelf 0.9 bug, but NixOS/patchelf#170 prevents us from upgrading to 0.10. I'd recommend just removing files from the aarch64 binary. As for splitting outputs, If I'm reading source correctly Hydra counts all outputs against the limit, which also makes quite sense. So we're back to deleting profiled builds. |
|
Looking at the options we can only pick between:
As an user I'd rather have debug symbols, so if there's a segfault I have something useful. If I want to profile my builds, then I'll have to do some extra work. |
|
I've pushed b930b2d that brings this a bit closer. Any objections disabling profiled builds on aarch64? |
|
Ah, that also brings down size of final ghc, so hopefully this should be fixed once |
|
Apparently not enough, but we got further than before. |
|
That's weird: |
|
You need to add |
|
Of course, sum of file sizes is just an approximation for real NAR size (which is what's measured here IIRC): |
This makes ghc output be lower than 2GB limit. Also enable aarch64 build parallelism now that it's fixed in GHC 8.6
|
This should be now fixed on master. |
|
@vcunat for store paths, you can also use |
|
According to https://gitlab.haskell.org/ghc/ghc/-/issues/17051 no one has investigated this issue upstream in the past 3 years... thus I'd like to gently advocate for just raising the 2GB limit if at all possible? I understand the general need of an upper limit but why a seemingly overly conservative/restrictive 2GB in 2023? Is there any reason behind 2GB or is it just an arbitrary number that was chosen once? Since this issue was opened Apple made aarch64 much more prominent and not being able to profile haskell programs on it is pretty bad. Is cost the primary reason of not wanting to raise the limit? If yes is it possible to do some simple impact analysis on the cost increase of eg. doubling it? |
|
The limit is 3 GiB for some time now. |
|
Ah, thanks @vcunat good to know! I guess the same question applies re how is this limit being decided on? My guess is some cost analysis but I have no idea. |
|
No. Very ad-hoc right now. We'd better limit the compressed size by some lower value instead of this, too. |
|
As for costs, these discussions are very related: |
|
I triggered a build yesterday, and the GHC derivation came back with In any case, what about filing a PR dropping that aarch64 conditional in and see how it goes? |
|
For what it's worth, as of 9.6.2 I see no appreciable difference in binary distribution sizes on x86-64 and AArch64: |
rrbutani
added
6.topic: llvm/clang
On master,
ghc.aarch64-linux, Hydra reports:and consequently we won't have any haskell packages for aarch64.
Yes, it's always been big and now we've apparently hit some limit, though I'm not even sure which part enforces it. I suppose we want to just increase it, though it might be possible to slim ghc a bit, too.
The text was updated successfully, but these errors were encountered: