-
-
Notifications
You must be signed in to change notification settings - Fork 13.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
simpler, saner cross-compilation #227327
Comments
Possible solution to the |
Note that the dependency attributes are used for more than that https://github.com/search?q=repo%3ANixOS%2Fnixpkgs%20targetOffset&type=code nixpkgs/pkgs/stdenv/generic/setup.sh Line 518 in 4f32285
|
From the peanut gallery I've only been using nix for about a year and a half now and have been continuously confused by the naming around cross-compiling so I think this is a good direction to go in. One thing that continuously trips me up is the name |
@kjeremy We're following the autoconf definitions gnu.org/software/autoconf/manual/autoconf-2.68/html_node/Specifying-Target-Triplets.html while you're thinking of something else |
initial work #227502 |
Yeah, autoconf made some less-than-ideal choices there (mainly "host") but we are sort of stuck with them. Trying to change the terminology creates even more confusion (cough llvm cough). I do think that nixpkgs makes the situation a bit worse by saying that non-code-emitting packages (like |
@Artturin thanks for pointing out nixpkgs/pkgs/stdenv/generic/setup.sh Lines 647 to 650 in e6899c3
nixpkgs/pkgs/stdenv/generic/setup.sh Lines 683 to 698 in e6899c3
Ugh. I sort of wish hooks didn't exist, at least as bash scripts... I wish they were managed at the level of Nix code and stitched together by My initial instinct is that we could stash these offsets in a
Awesome, will review. |
Another solution to the The example below is copied from this issue below the text "instead it could look like" since I can't seem to link directly to that example: pkgs:
pkgs.stdenv.mkDerivation (self: {
# deps here is a "local" variable, not passed to builtins.mkDerivation
deps = {
inherit (pkgs) pkg1 pkg2;
};
buildInputs = [ self.deps.pkg1 self.deps.pkg2 ];
}) The "let In the context of getting rid of splicing, this would mean pkgs:
pkgs.stdenv.mkDerivation (self: {
# deps here is a "local" variable, not passed to builtins.mkDerivation
deps = {
inherit (pkgs.onBuild) cmake makeWrapper;
inherit (pkgs.onBuildForHost) gfortran;
inherit (pkgs.onHost) openssl;
};
inputs = with self.deps; [ cmake makeWrapper gfortran openssl ];
}) With splicing gone and inputs = builtins.attrValues self.deps; I think it can, which means we can drop the |
@Ericson2314's issue #28327 |
This issue has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/is-this-the-current-state-of-things/45944/13 |
This issue has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/frustrations-about-splicing/49607/1 |
This issue has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/frustrations-about-splicing/49607/16 |
This is a long-term goal; none of this can happen overnight. This is also an incomplete work-in-progress.
TL;DR: Short Example
Wishlist
Eliminate splicing
Splicing is too magical. Probably three1 people really understand it. Dozens of people think they understand it but really don't, and get pissed off when it breaks in weird ways. This is a major part of why people hate on cross compilation.
Eliminating splicing likely means that package expressions will need to take package sets explicitly as top-level arguments. See next section.
Eliminate special exceptions to
depsFooBar
namingBasically, do this (thanks @Artturin):
targetPlatform==null
for packages that do not emit codeThe vast majority of packages in nixpkgs don't emit code. Instead of setting their
targetPlatform
equal to theirhostPlatform
, we should set it tonull
.Using
targetPlatform==hostPlatform
erases the distinction between packages that emit code for theirhostPlatform
and packages that don't emit code. It also creates ambiguity: these packages can be moved freely between (say)depsBuildHost
anddepsBuildTarget
with no change in behavior, so the difference betweendepsBuildHost
anddepsBuildTarget
is harder for people to learn.Prefix all binaries that have a targetPlatform
AKA resurrect this PR:
Right now we have an artificial distinction between:
pkgsBuildTarget
for build=X target=YpkgsHostTarget
for host=X target=YThe first kind won't have a
aarch64-linux-
prefix on their code-emitting binaries (like gcc). The latter kind will. This is silly. Prefix all the binaries, eliminate the distinction.For compilers (clang, rust) that have a single binary entry point for all platforms and use some kind of
--target=
flag, we simply wrap that entry point with a wrapper for eachtargetPlatform
which adds the--target=
flag.Use the pythonPackages
onBuildForHost
namingpkgsFooBar
is extremely unergonomic. Nobody can remember whatFoo
is for and whatBar
is for. These should be changed topkgsOnBuildForHost
likepythonPackages
does; withpkgsOnFooForBar
least that way there is a reminder of whatFoo
andBar
are:pkgsOnFooForBar
runs onFoo
and emits code forBar
.Packages with no target (see previous heading) get shorter names:
pkgsOnBuild
andpkgsOnHost
.No more
depsOnXForY
, onlypkgsOnXForY
The distinction between the various
depsFooBar
attributes of a derivation serves two purposes:depsBuildX
stuff goes in the$PATH
at build timeInstead of (six? more?) different derivation attributes we only need two:
Long Example
This example might look complicated, but it exercises all possible arguments and attributes. Think about that. This is the most complicated attrset you'll ever see.
Non-code-emitting packages will only have, at most, the first five attributes (one of which is never used in practice). In practice most packages will have only three arguments:
pkgsOnHost
,pkgsOnBuild
, andpkgsOnBuildForHost
.Problems
Note: there is a possible solution to both of the following two problems, which brings additional benefits.
This needs to be harmonized with
callPackage
. It is really unfortunate that the Nix language does not let functions declare deep attributes as arguments, like{foo.bar, ...}: ...
.In its current form this makes
.override
very painful to use, since it don't cope well with overriding a sub-attribute of an argument. It should be extended to provide an ergonomic way to override sub-attributes of an argument.Questions
Should
deps
anddepsInPATH
be combined into a single attribute?mkDerivation
could decide whether or not to put each dependency into the$PATH
based on whetherstdenv.buildPlatform.canExecute
that dependency'shostPlatform
.In practice there are very very very few situations where you don't want all the possibly-executable dependencies put into the path; we could provide an "escape hatch" to mark specific dependencies as "don't belong in the
$PATH
even if we could execute them".with
is a footgun, and this scheme encourageswith pkgsOnBuild
. We could makedeps
anddepsOnBuild
be attrsets (whose attrnames are ignored); this would allow the use of the much-saferinherit
syntax. A better long-term solution would be RFC 110.Should
pkgsOnBuildForHost
be an attribute within thepkgsOnBuild
attrset? For example,pkgsOnBuild.forHost
. This would mean only two arguments for non-compiler-like packages.Footnotes
I am not one of these people. But I'm also aware that I don't fully understand splicing. ↩
The text was updated successfully, but these errors were encountered: