-
-
Notifications
You must be signed in to change notification settings - Fork 13.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Discourage/prohibit inheriting from package sets in callPackage #204303
Comments
@Artturin , please, can you (or someone) explain what is the matter around the splicing? I have seen today some code on XFCE mega-attrset, and I am confused. Also, I have some packages that use Darwin libs, so I will start to cleanup them. |
splicing adds the
and
|
I think it is not possible to get rid of it entirely, especially if you consider the API benefit of being able to override individual inputs which is certainly an intention in some of the cases. Also, in sub package sets the solution you propose in #195874 won't work, of course. |
There's some magic going on with |
I concur that, ideally, we do not want to pass sets of packages to packages. Same reason we don't just pass Why are the Darwin frameworks hidden away in a subset anyways? If there's no good reason for it, we might want to consider just adding them to the top-level. Another thing we could consider is a special attribute where we can simply pass attribute sets from which attrs should be considered as if they were at the top-level but at that point, we might as well use the top-level in the first place? |
I don't believe they are hidden, properly speaking. Anyone can access them by the cumbersome inherit mechanism.
This is a bit personal and preciousness-y, but I do not like the currently huge top-level file and its propensity to scramble the asciibetical ordering. |
stuff like this should be discouraged/prohibited too keybinder = callPackage ../development/libraries/keybinder {
automake = automake111x;
lua = lua5_1;
};
if it has to be done then it should be keybinder = callPackage ../development/libraries/keybinder {
automake = __splicedPackages.automake111x;
lua = __splicedPackages.lua5_1;
};
|
can we fix that by changing (all-packages.nix L13) with pkgs; to with pkgs.__splicedPackages; ? |
i linked #58327 in the OP a thing we could try is reorganize all-packages.nix so we don't get infinite recursion quick POC (added with pkgs.__splicedPackages under wrapBintoolsWith which was giving inf rec) diff --git a/pkgs/top-level/all-packages.nix b/pkgs/top-level/all-packages.nix
index 5e0e798be20..765ce0b92c0 100644
--- a/pkgs/top-level/all-packages.nix
+++ b/pkgs/top-level/all-packages.nix
@@ -8,7 +8,7 @@
{ lib, noSysDirs, config, overlays }:
res: pkgs: super:
-with pkgs;
+(with pkgs;
{
# A module system style type tag
@@ -15618,6 +15618,9 @@ with pkgs;
yaml-language-server = nodePackages.yaml-language-server;
+}) // (with pkgs.__splicedPackages; {
+
+
# prolog
yap = callPackage ../development/compilers/yap { };
@@ -38170,4 +38173,4 @@ with pkgs;
aitrack = libsForQt5.callPackage ../applications/misc/aitrack { };
widevine-cdm = callPackage ../applications/networking/browsers/misc/widevine-cdm.nix { };
-}
+}) a issue with that is the use of // so we won't get errors about duplicate attrs and the RHS will override the LHS |
That's what I mean.
Note that I said "top-level", not "top-level file". I don't care what file they're in but I do want them to be accessible from Alternatively, an extra attr that automagically extends the main package set for the purpose of discovering dependencies for that specific package would also be fine in my eyes. i.e. bar = callPackage ../foo/bar {
extraPackageSets = [ darwin xorg ];
} {
stdenv,
libX11,
Security,
...
}
...
In cases like this, I prefer to simply reference {
lib,
stdenv,
automake111x,
lua5_1,
...
}:
let
lua = lua5_1;
automake = automake111x;
in
stdenv.mkDerivation {
nativeBuildInputs = [ automake lua ];
} (Or just in-line without the let.) IMO, we should have as little logic as possible in |
It's becoming confusing... It should be something more, ehr, ergonomical. Yes, I believe we should be more cross-compilaton-aware and we should strive to make it a "first class citizen" in Nix{,pkgs}. Or maybe there is a terminology we all should learn? |
The most ergonomic will always be to not have package sets and have everything under the top-level as I initially suggested. My alternative suggestion is just something that came to my mind because we might need something like that anyways because we do have huge package sets whose packages other packages depend on and, unlike the smaller ones we've been discussing so far, we probably don't want to dump those into the top-level. |
Taking the idea to a ridiculous extreme, all the expressions organized in subdirectories can be "concatenated" in a huge 4GB single file, sqlite-amalgam style. Thinking a bit about it, this makes few to no sense. We have custom code to shelter platforms like FreeBSD and PowerPC, it would be insane to make all of it accessible to the top-level. Edit: I found the word: uniformization! |
The frameworks come from different sdks, so sometimes we want the ones from apple_sdk, sometimes apple_sdk_11_0 (and there will be more). |
In a certain sense, yes. |
According to the issue linked at the end, filling arguments for `callPackage` in all-packages.nix breaks splicing and makes the expressions incompatible with cross-compilation. This issue is more pronounced in the many `inherit (darwin)` calls. However, it is present in similar contexts too. The idea here is not the smartest: it just hides those inheritances from `all-packages.nix` and sends them to the `default.nix` expressions. Issue: NixOS#204303
According to the issue linked at the end, filling arguments for `callPackage` in all-packages.nix breaks splicing and makes the expressions incompatible with cross-compilation. This issue is more pronounced in the many `inherit (darwin)` calls. However, it is present in similar contexts too. The idea here is not the smartest: it just hides those inheritances from `all-packages.nix` and sends them to the `default.nix` expressions. Issue: NixOS#204303
According to the issue linked at the end, filling arguments for `callPackage` in all-packages.nix breaks splicing and makes the expressions incompatible with cross-compilation. This issue is more pronounced in the many `inherit (darwin)` calls. However, it is present in similar contexts too. The idea here is not the smartest: it just hides those inheritances from `all-packages.nix` and sends them to the `default.nix` expressions. Issue: NixOS#204303
Don't inherit from package sets in all-packages NixOS#204303 Fix update script to use new hash cargoHash attrs
@fgaz I have been working on something similar. It's not finished, but I've posted the work-in-progress before I factor in your Thanks to @Artturin for letting me know about this. |
Note that RFC 140 will deprecate |
@amjoseph-nixpkgs No it won't deprecate that, such invocations will still be valid! It's in scope to look at for the Nixpkgs Architecture Team, but until custom arguments have an adequate replacement of its use cases (which RFC 140 isn't), it shouldn't be deprecated |
I've experimented with a similar approach that doesn't require changes to the Nix language. In master...AtnNn:nixpkgs:deepcall |
@AtnNn very nice, thanks for trying it out! While yours is a clever solution given the current Nix language limitation, I think in the long run a feature like the one I proposed is necessary to keep everything clean. Nix is a domain specific language after all! At the moment I'm busy, but I might draft an RFC later this year |
What can Missing arguments are the only thing I can think of, and the easy workaround is to add |
Thanks for remembering me about this! |
This issue has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/is-this-the-current-state-of-things/45944/14 |
Random suggestion - how about implementing cross via an argument in |
I'm not sure how you're imagining this to work with overrideAttrs? |
We'd add Edit: Nevermind, I forgot about all of the variables defined outside the |
This issue has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/frustrations-about-splicing/49607/1 |
This issue has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/frustrations-about-splicing/49607/7 |
The way forward for Darwin is to get rid of frameworks and provide an SDK in the stdenv. It doesn’t help other package sets, but it should at least get rid of one of the major contributors to this problem. |
in all-packages.nix when something is inherited (ex,
{inherit (pkgs.darwin.apple_sdk.frameworks) Security;}
) in acallPackage
invocation the inherited attr will not have__spliced
and so won't be cross-compilation compatible.one way to fix this would be to change the
with pkgs;
in all-packages towith pkgs.__splicedPackages
like tried in #58327however that will result in other issues like infinite recursion.
other way to work around it would be to to inherit from the
__splicedPackages
set (ex,{inherit (pkgs.__splicedPackages.darwin.apple_sdk.frameworks) Security;}
)the way i prefer would be to just not inherit from package sets
Demo:
show-splices.nix
working:
broken:
related issues:
#49526
The text was updated successfully, but these errors were encountered: