[RFC 0140] Simple Package Paths #140
Conversation
|
This gets squash merged, right? |
|
@SuperSandro2000 I believe it does get squashed. |
|
I didn't read it fully but IMO we should link the tooling already developed to prove that the transition is easy and doable. |
Originally we wanted to maintain the commit history from the original repository, but it's probably better to just squash it beforehand, which I now did.
It does link to NixOS/nixpkgs#211832, which itself links to the (yet unfinished) tooling (also linked in the RFC description). I think that's good enough, especially since the tooling might move around. That PR does evaluate cleanly in CI though, which is already fairly good as a proof. I also have the time to finish that tooling to be fully in line with the RFC. |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/nixpkgs-architecture-team-meeting-27-rfc-time/25095/1 |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/tweag-nix-dev-update-43/25185/1 |
rfcs/0140-simple-package-paths.md
Outdated
|
|
||
| - Use `unit` (at the Nixpkgs root) instead of `pkgs/unit`. | ||
| This is future proof in case we want to make the directory structure more general purpose, but this is out of scope | ||
| - Other name proposals were deemed worse: `pkg`, `component`, `part`, `mod`, `comp` |
There was a problem hiding this comment.
I would like to nominate auto, to me this is more future-proof than unit so we can include multiple packages in one directory
There was a problem hiding this comment.
so we can include multiple packages in one directory
How do you mean that?
There was a problem hiding this comment.
For example, we currently have element-desktop and element-web that share the same pin.json (or generic.nix in some other cases), in the future we might want to allow that to be defined in pkgs/unit, but I'm not quite sure the name unit fits the behavior of having multiple packages under the same directory, and auto is more future-proof in that case IMO
There was a problem hiding this comment.
I see, yeah so this has been discussed before. I believe in the team the consensus was that multiple versions and variants should eventually end up in the same pkgs.element attribute, once we have the design for that figured out. In that case, all those variants would naturally be together in the pkgs/unit/el/element directory.
That's also kind of why unit was chosen, we talked about explaining it better before, something like:
unit: A collection of standardized files related to the same software component
There was a problem hiding this comment.
that makes sense to me, thanks for the explanation
There was a problem hiding this comment.
Naming is really important, especially for something as ubiquitous. Please add the rationale for choosing "unit" over the other proposals, just as you did for all the other decisions. Otherwise it will produce guesswork, confusion, and needless resistance in readers.
There was a problem hiding this comment.
I don't dislike "auto" but I'd like to note that the name implies the package is automatically called. This means that packages with custom arguments would not fit in there anymore, unlike with the current proposal. Maybe they'd be put into "pkgs/manual" instead then? (which then opens up to confusion with manual as in documentation, etc.)
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/nix-looking-back-at-2022/25234/1 |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/2023-01-06-nixpkgs-architecture-team-meeting-28/25262/1 |
|
How would (e.g.) Emacs and its ecosystem would behave on this new system? |
| - (-) Not using `default.nix` frees up `default.nix` for an expression that is actually buildable, e.g. `(import ../.. {}).hello`, although we don't yet have a use case for this that isn't covered by `nix-build ../.. -A <attrname>`. | ||
| - (-) Using `default.nix` would tempt users to invoke `nix-build .`, which wouldn't work and making package functions auto-callable is a known anti-pattern. |
There was a problem hiding this comment.
I doubt people are actually doing that and this is something to worry about. Everyone I have observed and seen goes though all-packages.nix except when doing out of tree things which doesn't apply here anyway.
There was a problem hiding this comment.
I doubt people are actually doing that and this is something to worry about. Everyone I have observed and seen goes though all-packages.nix except when doing out of tree things which doesn't apply here anyway.
A additional benefit of package.nix is that if one day nixos module files are put in the same dir as the package then there will be package.nix and module.nix
There was a problem hiding this comment.
This RFC has been accepted & merged after extensive discussion several months ago, so I might be a bit late to add this input here. As I understand the process, the correct way would be to create a new RFC to suggest your desired change if you care strong enough about it.
There was a problem hiding this comment.
Indeed, and you already brought this up before here
| Alternatives: | ||
| - `default.nix` | ||
| - (+) `default.nix` is already a convention most people are used to. | ||
| - (-) We don't benefit from the usual `default.nix` benefits: |
There was a problem hiding this comment.
We would benefit that people are used to placing default.nix files in directories. It is the defacto filename which IMO should have just continued to be used.
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/2023-10-31-nixpkgs-architecture-team-meeting-45/34846/1 |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/2023-11-28-nixpkgs-architecture-team-meeting-46/36171/1 |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/ci-will-soon-enforce-pkgs-by-name-for-new-packages/38098/1 |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/tweag-nix-dev-update-52/38343/1 |
ghost
mentioned this pull request
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/nixpkgs-architecture-team-conclusion-and-prospective/41020/1 |
* RFC 140 Initialized from nixpkgs-architecture/simple-package-paths@01948e0 * Minor improvements * Minor nits * Update co-authors and add pre-RFC reviewers * pkg-fun.nix -> package.nix * Mid-sized refactor for improved clarity and incorporating feedback In addition to some more minor changes and incorporating feedback, the major changes are: - Restructure the RFC into two separate parts, one to introduce the convention and one to migrate packages to it when possible - Remove the restriction that files inside a unit directory can only be referenced by the corresponding `pkgs.${name}`. It feels very unnatural to have this restriction and it's hard to reason about it. Files inside a unit directory still can't reference anything _outside_ the unit directory, which is very similar to Nix's concept of allowed-uris, which may be used to implement this check in the future. - Remove the special case of allowing custom arguments. By not having this one exception, users viewing a unit directory can be sure that there's no hidden semantics anywhere (overriding arguments) and that the functions arguments correspond directly to attributes in `pkgs`, no exceptions that would require looking at `all-packages.nix`. And it would be weird just to allow this one exception of `callPackage` with custom arguments, when there's a lot of other similarly small exceptions we could make, like allowing `python3Packages.callPackage`. - Remove the requirement that new packages must use this standard. Especially with the above exception removed, this standard is now more strict and less packages satisfy it by default. A scenario could be that a user adds a new package, initially not needing custom arguments, so CI requires it to be in `pkgs/unit`, but then a custom argument is needed, so it must be moved out of there and added to `all-packages.nix`. But then the custom argument can be removed, so it _must_ be in `pkgs/unit` again. This sucks. So let's keep `all-packages.nix` unrestricted, so a package won't have to be moved back and forth like this. * Re-add custom argument exception and various minor improvements * shard distribution stats, cleanup, more uniformity * Readd accidentally removed definition * Mention package variants * Minor moving and formatting * Changes from feedback in the meeting * Link to demonstration of cherry-picking without problems * Link to demonstrates of problematic/non-problematic Git operations * Names must be unique when lowercased * Properly close invisible anchor * Update summary to mention Nixpkgs more explicitly * Include more arguments and counter-arguments for pkgs/unit alternatives * Add shepherd team and nicks * Convert frontmatter to a table * Fix table rendering * Minor fixups Co-Authored-By: Robert Hensing <[email protected]> * Explain unit and add more alternatives * unit -> by-name, remove "standard" And some very minor changes * Apply suggestions from code review Remove the barely used term "base directory" Co-authored-by: Robert Hensing <[email protected]> --------- Co-authored-by: Robert Hensing <[email protected]> Co-authored-by: Robert Hensing <[email protected]>
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: |
…d-support As a consequence of restrictions imposed by RFC 140 - Simple Package Paths [1] -, files related to a package should be confined on the package directory. Certainly this restriction does not apply to packages outside by-name hierarchy. Nonetheless, this is an interesting organization heuristics: things that affect Emacs should be confined inside Emacs directory. Besides a future migration, the "debuggability" of a framework is way more enhanced when we know how to find all its files. A similar task was done before, when RFC 140 was not a thging yet - namely, the migration of emacs-modes to elisp-packages [2]. [1] NixOS/rfcs#140 [2] NixOS#123859
…d-support As a consequence of restrictions imposed by RFC 140 - Simple Package Paths [1] -, files related to a package should be confined on the package directory. Certainly this restriction does not apply to packages outside by-name hierarchy. Nonetheless, this is an interesting organization heuristics: things that affect Emacs should be confined inside Emacs directory. Besides a future migration, the "debuggability" of a framework is way more enhanced when we know how to find all its files. A similar task was done before, when RFC 140 was not a thging yet - namely, the migration of emacs-modes to elisp-packages [2]. [1] NixOS/rfcs#140 [2] NixOS#123859
Summary
Auto-generate trivial top-level attribute definitions in Nixpkgs'
pkgs/top-level/all-packages.nixfrom a Nixpkgs-internally standardised directory structure that matches the attribute name.This makes it much easier to contribute new packages, since there's no more guessing needed as to where the package should go, both in the ad-hoc directory categories and in
pkgs/top-level/all-packages.nix.Rendered
Implementation:
pkgs/by-name: Enforce for new packages nixpkgs#275539pkgs/by-name: Automated migration nixpkgs#211832Nixpkgs Architecture Team
This RFC has been worked on openly in a repository by the new Nixpkgs Architecture Team in the last couple months. It is the first of hopefully many quality of life improvements to nixpkgs.
The team will continue meeting regularly (calendar), see here for more details, anybody is free to join and listen in or ask questions. Also feel free to join the Matrix room to discuss this RFC.
Shepherd team
Here are the shepherds with their nominations, all of which are now accepted.
Major RFC updates