From 921c7269e69bcc9c99f871e46d76b44fc1db2a92 Mon Sep 17 00:00:00 2001 From: Anderson Torres Date: Sun, 23 Apr 2023 00:59:56 -0300 Subject: [PATCH] Meta.Categories, not Filesystem Diretory Trees --- rfcs/0146-meta-categories.md | 204 +++++++++++++++++++++++++++++++++++ 1 file changed, 204 insertions(+) create mode 100755 rfcs/0146-meta-categories.md diff --git a/rfcs/0146-meta-categories.md b/rfcs/0146-meta-categories.md new file mode 100755 index 000000000..fa15a3223 --- /dev/null +++ b/rfcs/0146-meta-categories.md @@ -0,0 +1,204 @@ +--- +feature: Decouple filesystem from categorization +start-date: 2023-04-23 +author: Anderson Torres +co-authors: (find a buddy later to help out with the RFC) +shepherd-team: (names, to be nominated and accepted by RFC steering committee) +shepherd-leader: (name to be appointed by RFC steering committee) +related-issues: (will contain links to implementation PRs) +--- + +# Summary +[summary]: #summary + +Deploy a new method of categorization for the packages maintained by Nixpkgs, +not relying on filesystem idiosyncrasies. + +# Motivation +[motivation]: #motivation + +Currently, Nixpkgs uses the filesystem, or more accurately, the directory tree +layout in order to informally categorize the softwares it packages, as described +in the [Hierarchy](https://nixos.org/manual/nixpkgs/stable/#sec-hierarchy) +section of Nixpkgs manual. + +This is a simple, easy to understand and consecrated-by-use method of +categorization, partially employed by many other package managers like GNU Guix +and NetBSD pkgsrc. + +However this system of categorization has serious problems: + +1. It is bounded by the constraints imposed by the filesystem. + + - Restrictions on filenames, subdirectory tree depth, permissions, inodes, + quotas, and many other things. + - Some of these restrictions are not well documented and are found simply + by "bumping" on them. + - The restrictions can vary on an implementation basis. + - Some filesystems have more restrictions or less features than others, + forcing an uncomfortable lowest common denominator. + - Some operating systems can impose additional constraints over otherwise + full-featured filesystems because of backwards compatibility (8 dot + 3,anyone?). + +2. It requires a local checkout of the tree. + + Certainly this checkout can be "cached" using some form of `find . > + /tmp/pkgs-listing.txt`, or more sophisticated solutions like `locate + + updatedb`. Nonetheless such solutions still require access to a fresh, + updated copy of the Nixpkgs tree. + +3. The creation of a new category - and more generally the manipulation of + categories - requires an unpleaseant task of renaming and eventually patching + many seemingly unrelated files. + + - Moving files around Nixpkgs codebase requires updating their forward and + backward references. + - Especially in some auxiliary tools like editor plugins, testing suites, + autoupdate scripts and so on. + - Rewriting `all-packages.nix` can be error-prone (even using Metapad) and it + can generate huge, noisy patches. + +4. There is no convenient way to use multivalued categorization. + + A piece of software can fulfill many categories; e.g. + - an educational game + - a console emulator (vs. a PC emulator) + - and a special-purpose programming language (say, a smart-contracts one). + + The current one-size-fits-all restriction is artificial, imposes unreasonable + limitations and results in incomplete and confusing information. + + - No, symlinks or hardlinks are not convenient for this purpose; not all + environments support them (falling on the "less features than others" + problem expressed before) and they convey nothing besides confusion - just + think about writing the corresponding entry in `all-packages.nix`. + +5. It puts over the (possibly human) package writer the mental load of where to + put the files on the filesystem hierarchy, deviating them from the job of + really writing them. + + - Or just taking the shortest path and throw it on a folder under `misc`. + +6. It "locks" the filesystem, preventing its usage for other, more sensible + purposes. + +7. The most important: the categorization is not discoverable via Nix language + infrastructure. + + Indeed there is no higher level way to query about such categories besides + the one described in the bullet 2 above. + +In light of such a bunch of problems, this RFC proposes a novel alternative to +the above mess: new `meta` attributes. + +# Detailed design +[design]: #detailed-design + +A new attribute, `meta.categories`, will be included for every Nix expression +living inside Nixpkgs. + +This attribute will be a list, whose elements are one of the possible elements +of the `lib.categories` set. + +A typical snippet of `lib.categories` will be similar to: + +```nix +{ + assembler = { + name = "Assembler"; + description = '' + A program that converts text written in assembly language to binary code. + ''; + }; + + compiler = { + name = "Compiler"; + description = '' + A program that converts a source from a language to another, usually from + a higher, human-readable level to a lower, machine level. + ''; + }; + + font = { + name = "Font"; + description = '' + A set of files that defines a set of graphically-related glyphs. + ''; + }; + + game = { + name = "Game"; + description = '' + A program developed with entertainment in mind. + ''; + }; + + interpreter = { + name = "Interpreter"; + description = '' + A program that directly executes instructions written in a programming + language, without requiring compilation into the native machine language. + ''; + }; + +``` + +# Examples and Interactions +[examples-and-interactions]: #examples-and-interactions + +In file bochs/default.nix: + +```nix +stdenv.mkDerivation { + +. . . + + meta = { + . . . + categories = with lib.categories; [ emulator debugger ]; + . . . + }; + }; +} + +``` + +# Drawbacks +[drawbacks]: #drawbacks + +The most immediate drawbacks are: + +1. A huge treewide edit of Nixpkgs + + On the other hand, this is easily sprintable and amenable to automation. + +2. Bikeshedding + + How many and which categories we should create? Can we expand them later? + +# Alternatives +[alternatives]: #alternatives + +1. Do nothing + +2. Ignore/nuke the categorization completely + + This is not an idea as bad as it appear. After all, categorization has a + non-negligible propensity to bikeshedding. Removing it removes all problems. + + Nonetheless, other good software collections do this just fine, and we can + easily imitate them. Indeed, we can follow/take a peek at how Repology keeps + the categorizations defined by those software collections. + +# Unresolved questions +[unresolved]: #unresolved-questions + +Still unsolved is what data structure is better suited to represent a category. + +# Future work +[future]: #future-work + +- Curation of categories. +- Update documentation. +