lib.extendMkDerivation, lib.adaptMkDerivation: init

[ShamrockLee]

Description of changes

This PR introduces a unified approach to implement build helpers that support fixed-point arguments (lib.extendMkDerivation) and bring such support to existing build helpers (lib.adaptMkDerivation).

The fixed-point arguments support in stdenv.mkDerivation is introduced in #119942, and there are ongoing attempts to make other build helpers support such syntax (buildRustPackage refactor #194475, buildGoModule refactor #225051). The overlay style overrideAttrs brought by the stdenv.mkDerivation change can be used to implement the functionality, which is adopted by the buildRustPackage refactor and the previous version of the buildGoModule refactor. The challenge of such an approach is that the whole set pattern matching the input arguments are degenerated into a single identifier, making it hard to see from the source code which attributes to the build helper accepts.

The new Nixpkgs Library function, lib.extendMkDerivation accepts a base build helper and an attribute overlay (an overlay in the form finalAttrs: args: <attrsToUpdate>), and returns a new build helper by extending the base build helper with the attribute overlay via its <pkg>.overrideAttrs.

The following is the definition of an example build helper, mkLocalDerivation:

lib.extendMkDerivation { } stdenv.mkDerivation (
  finalAttrs:
  {
    preferLocalBuild ? true,
    allowSubstitute ? false,
    ...
  }@args:

  # No need of `// args` here.
  # The whole set of input arguments is passed in advance.
  {
    # Attributes to update
    inherit preferLocalBuild allowSubstitute;
  }
)

For existing build helpers with arguments that cannot be passed to the base build helper, another Nixpkgs library function, lib.adaptMkDerivation, is provided. Instead of an attribute overlay, it takes a function that turns the arguments of the result build helper to those of the base build helper, known as an argument set adapter. This way, developers can decide not to pass some arguments down to the base build helper.

lib.adaptMkDerivation { } stdenv.mkDerivation (
  finalAttrs:
  {
    preferLocalBuild ? true,
    allowSubstitute ? false,
    specialArg ? (_: false),
    ...
  }@args:

  removeAttrs [
    # Don't pass specialArg into mkDerivation
    "specialArg"
  ] args
  // {
    # Arguments to pass
    inherit preferLocalBuild allowSubstitute;
    # Some expressions involving specialArg
    greeting = if specialArg "hi" then "hi" else "hello";
  }
)

The reason to have two functions instead of just lib.adaptMkDerivation, is that we want to encourage (new) build helpers to pass all their input arguments properly down to stdenv.mkDerivation, and provide override functionality through the standardized <pkg>.overrideAttrs instead of custom overriders such as overridePythonAttrs, overrideNimAttrs, etc, by encouraging the use of lib.extendMkDerivation. In the meantime, providing lib.extendMkDerivation-incompatible build helpers with fixed-point arguments support (by lib.adaptMkDerivation) enables a smoother transition to a lib.extendMkDerivation-compatible style.

Fewer expression changes is another plus to convert existing build helpers with lib.adaptMkDerivation to take fixed-point arguments. Working build helper to support recursive attributes. A subsequent PR #271387 enables the fixed-point arguments support in buildPythonPackage and buildPythonApplication with slight modification against mk-python-derivation.nix with no rebuilds.

Both lib.extendMkDerivation and lib.adaptMkDerivation take a set of optional arguments to manipulate their behaviors. For example lib.adaptMkDerivation { modify = drv: toPythonModule drv; } applies toPythonModule to the derivation produced by the derived build helper.

Cc:
Python maintainers: @FRidh @mweinelt @jonringer
Author of the buildRustPackage refactor PR @amesgen
Reviewer of the buildGoModule refactor PR @zowoq
Author of the merged recursive stdenv.mkDerivation PR @roberth
People who mention 119942 in Python application definition: @LunNova

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 23.05 Release Notes (or backporting 22.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

[ShamrockLee]

I should have checked more carefully.

There's a function called makeOverride that deprives the input builder of their recursive attributes support, and buildPythonPackage and buildPackageApplication happens to be makeOverrided. While the change keeps the current python packages unchanged, it still doesn't bring the recursive attributes support to buildPythonModule and buildPythonApplication unless we fix makeOverridable.

Updat: fixed, see below.

[ShamrockLee]

Fix lib.makeOverridable and makeOverridablePythonPackage. Now buildPython* accepts recursive attributes.
Turn a Python application pyspread into the recursive attributes style.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/proposal-move-values-from-derivation-attributes-to-function-arguments/20697/21

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/avoid-rec-expresions-in-nixpkgs/8293/18

[roberth]

A mix of incremental and fundamental suggestions, because I can't decide for the nixpkgs-python maintainers.

I'm concerned about the increasing complexity, while the goal can be achieved through simplification instead: removing the python-specific level of overriding and turning it into an "overlay" on the mkDerivation arguments instead.

The alternative strategy is to

1. Provide this python layer, which contains the relevant mkPyton* logic in a way that works with overlay-style overriding. This can be done by reading the existing code, attribute for attribute, and adding the logic to the python layer.
2. Change the python packages to use that layer in combination with mkDerivation, instead of the current mkPython* functions.
3. Perhaps make the mkPython functions reuse the overlay so that they don't literally reimplement the same logic. I don't know if this would be worthwhile.
4. Eventually deprecate the mkPython* functions.

I have played around with step 1 of the alternative strategy. It is feasible, but it requires a bit of migration for each package. I'm not a nixpkgs-python maintainer, so I don't feel like I should be the one to make the decision whether to accept the complexity of this PR, or refactor to actually simplify the python logic by fitting it into a mkDerivation layer.

[roberth]

It's worth noting that finalAttrs contains the final arguments to mkDerivation attrs, and not the final arguments to the function you're constructing.

[ShamrockLee]

Now the arguments after applying the modifier function is also considered.

IMO, the finalAttrs is meant to be the final state of the attributes passing to the base builder (mkDerivation). That's how user could access finalAttrs.finalPackage and other goodies. If we just want to get the input argument set manually, the user could just lib.fix the function themselves.

Input arguments that doesn't mean to be passed to the base builder are subject to special care, and should never enter the recursion. Those not-to-pass arguments are also the reason why builder overlays are not drop-in replacement of current builders.

[ShamrockLee]

I tried to add those special arguments to finalAttrs, but that greatly increase the complexity of the code and makes behavior surprising (the specified result will be different from what is got through finalAttrs by design).

A better way would be to encourage passing all the arguments. When all the arguments are passed, they will all be available inside finalAttrs, and we could then switch to the overlay-based workflow specification (from the current, build-helper-based one).

[roberth]

This feels too custom or repeated (not sure yet which). Does this add mkDerivation-like fixpoint logic at the python attrs level?

I believe we should merge the python attrs level into the mkDerivation attrs, so that the interface and implementation become simpler. Having multiple levels of overriding has a huge complexity cost, so getting rid of an unnecessary level would be a huge win. We'd get rid of overridePythonAttrs and all the user facing complexity, implementation complexity and bugs that come with it.

The python-specific attrs can almost be implemented as an "overlay" on the mkDerivation arguments. When I tried this, I think only like 3 attributes had the same name but a slightly different meaning. That made it a breaking change, but migrating those attributes is feasible and would vastly simplify the python/mkDerivation wiring.

[FRidh]

With same names that is generally speaking because the Python builder will extend the lists with some "defaults". That's really the only value of the custom builder over just plain hooks.

[ShamrockLee]

This feels too custom or repeated (not sure yet which). Does this add mkDerivation-like fixpoint logic at the python attrs level?

I personally prefer passing most of the attributes into mkDerivation, encourage the use of overrideAttrs and gradually deprecates overridePythonAttrs as well as other builder-specific override methods.

That will be a mass rebuild, so I just work around the makeOverridablePythonPackage obstacle in order to demonstrate the possibility to add the recursive attributes support without rebuild.

[infinisil]

Recently the Packages Modules Working Group started investigating whether something like the module system could be used to evaluate packages. We're tracking all work in this repository, meeting weekly, but working mostly asynchronously. It would be great if you could join the Matrix room of the WG and chat with us, or even join the team yourself to work on such issues!

[ShamrockLee]

Thank you for taking time reviewing this!

I'm concerned about the increasing complexity, while the goal can be achieved through simplification instead: removing the python-specific level of overriding and turning it into an "overlay" on the mkDerivation arguments instead.

The idea to shift workflow-specific overlays sounds neat, and that could also be friendlier when packaging multi-language projects. Nevertheless, there are some issue on the way to the switch:

1. Current builders rely more or less on arguments they won't pass into mkDerivation attribute set.
   Developers would need to change the way they handle the arguments / attributes, either by passing them directly or through passthru. The impl input argument of the proposed function is essentially an overlay that allows passing not-to-pass-to-mkDerivation arguments through the previous set.

2. Current attempt to try to implement a builder that supports the (finalAttrs: { }) through an overlay passing to .overrideAttrs, such as rustPlatform.buildRustPackage: support finalAttrs style #194475, discards the set pattern of input arguments ({ a, b ? "some default", ... }@args) completely and degenerate it into previousAttrs.
   It is a pity, since the set patterns itself serves as an interface of the builder, providing information about the expected input arguments to both and the Nix interpreter and people reading the Nix expression.
   The solution would be rather simple -- replace the previousAttrs with the set pattern, and then the remaining issue will be point 1.

Overall, the goal of the proposed function is to add (finalAttrs: { }) input support with minimum changes to the existing expressions, and raise the discussion about general approaches to bring such supports to builders.

[fricklerhandwerk]

Suggested change

	A Builder, in the context of Nixpkgs, are functions that produces derivations. (Don't confuse it with the `builder` input argument of function `derivation`, which refers to the path to the executable that builds the derivation.) Such function is usually designed to follow a specific workflow, so that one can produce a package by specifying a few options instead of messing up with the `derivation` function.
	A *builder*, in the context of Nixpkgs, is a function that produces derivations.
	:::{.warning}
	This is not to be confused with the [`builder` argument to `derivation`](https://nixos.org/manual/nix/unstable/language/derivations.html), which refers to the executable that produces the build result.
	:::
	Such a builder is usually designed to follow a specific workflow, so that one can produce a package by setting a limited set of options relevant to the particular use case instead of using the `derivation` function directly.

This is terrible naming. The least we can do is highlight it with red alarms (not sure about admonition syntax, it's all different everywhere...) Since you're going at great lengths to improve documentation (which is absolutely awesome!) we may as well change that while at it. I'd certainly support it by bouncing around proposals. But of course feel free to ignore requests to widen scope.

[ShamrockLee]

Possible candidates:

• Build function

  • Existing use referring to the same thing

    • androidenv.buildApp doc/languages-frameworks/android.section.md ## Building an Android application {#building-an-android-application}

      This build function is particularly useful when it is desired to use
      Hydra: the Nix-based continuous integration solution
      to build Android apps.

    • xcodeenv.composeXcodeWrapper doc/languages-frameworks/ios.section.md ## Deploying a proxy component wrapper exposing Xcode {#deploying-a-proxy-component-wrapper-exposing-xcode}

      The first use case is deploying a Nix package that provides symlinks to the Xcode
      installation on the host system. This package can be used as a build input to
      any build function implemented in the Nix expression language that requires
      Xcode.

  • Existing use referring to something else

    • Inside the build process of pkgs/build-support/rust/build-rust-crate/build-crate.nix

      # configure & source common build functions

• Build helper

  • Existing use referring to the same thing

    • buildIntegration pkgs/tools/networking/dd-agent/integrations-core.nix

      # Build helper to build a single datadog integration package.

  • Existing use referring to something else

    • Inside the meta.description of the Haskell Hackage package hein pkgs/development/haskell-modules/hackage-packages.nix

      An extensible build helper for haskell, in the vein of leiningen

The different use of both candidates doesn't seem to cause confusion. We could just choose from one of them, or come up with something else.

[fricklerhandwerk]

Great research. I like "build helper" more, because its closest to what the thing is about, and also "build function" only contains one word that adds information (almost everything is a function).

When we change section titles we should make sure URLs stay stable. Either keep the anchors or add redirects (we discussed this with @pennae somewhat recently but I can't remember why the script used in the Nix manual was not included).

[pennae]

we discussed this with @pennae somewhat recently but I can't remember why the script used in the Nix manual was not included

there's not technical reason it couldn't be done, it's just that nobody has done the work yet. adding first-class support for fragment redirects to nixos-render-docs would be nicer and less error-prone, but we don't have to do that as a first step.

[FRidh]

I disagree that we should rename this from builder to something else and that "build helper" is better. Quite the contrary, the function when called actually builds something, which I'd say is exactly what a builder is. This naming has been around for a very long time.

The Nix attribute builder is a detail that no typical user would ever encounter. Those interested going through say the Pills or in detail through the Nix manual will see it, but otherwise unless you're interested in writing a custom stdenv.mkDerivation you won't do anything with it. Hence, I don't think that part should even be mentioned here.

[fricklerhandwerk]

Nix language functions don't build anything, not even the derivation primitive. What we typically mean by "build" is to run some process to produce files from other files.

[FRidh]

Following that line of reasoning "build helper" is also incorrect, because we're solely preprocessing before deriving. So in that case "deriver" or "deriver helper" or "derivation preprocessor" or "build planner" is more correct. Though actually they invoke derivation as well so they're not solely peprocessors either.

[fricklerhandwerk]

Clearly this needs discussion, but I don't want to block this PR on a single term. I'm happy with "build helper" because it reduces ambiguity and is not obviously wrong, and I'm not opposed to further improving the terminology in a separate PR.

[ShamrockLee]

Maybe we could move the builder naming to another issue or PR.

BTW, "builders" also refers to "build machines" in the context of "remote builders", which is probably why the documentation of darwin.builder is misplaced under the "Builders" part in the Nixpkgs manual. It would be great to find a more suitable place for it. (#235858)

[infinisil]

Did we ever have a user be confused about these being called builders? I at least haven't, and I don't think we should be even spending time discussing this unless there are users confused about it.

[fricklerhandwerk]

Where do we find information about extendDerivation?

[ShamrockLee]

lib.customisation is currently not presented in the Nixpkgs manual. We'll need to add the documentation for those functions before referring to them.

[ShamrockLee]

Besides, the Nixpkgs Library Functions documentation automatically generated from the comment don't have anchors. So there's no way to link against them so far.

[ShamrockLee]

My fault. The anchor in the form function-library-<long attribute path> is also automatically generated. E. g. function-library-lib.customisation.extendMkDerivation.

This usage is very hard to discover, as the documentation is missing, and the manual provides no hyperlink in the title of each function document.

[ShamrockLee]

I addressed this review in the changes to add a new section about using lib.extendMkDerivatien to define build helpers. Please take a look.

[LunNova]

Just reviewing your change to input-remapper, I like how much this cleaned up its definition. :)

[jian-lin]

Does not compose well.

There are three build helpers for Emacs lisp packages: melpaBuild, elpaBuild and trivialBuild. These three build helpers all share one base build helper: genericBuild.

let
  genericBuild = adaptMkDerivation stdenv.mkDerivation (finalAttrs: { ... }@args: { ... })
in
melpaBuild = adaptMkDerivation genericBuild (finalAttrs: { ... }@args: { ... })

Use melpaBuild as an example. It receives arguments of its own like files and arguments of genericBuild like packageRequires. So ideally functionArgs melpaBuild should contains these two kinds of arguments.

[ShamrockLee]

Currently, the __functionArgs of derived build helpers don't include those from their based build helpers due to the nature of lib.functionArgs.

It's possible to add it as a new feature, which would be great for extendMkDerivation. However, it might not be a good idea for adaptMkDerivation- derived build helpers to inherit __functionArgs from the base build helper, as the primary goal of adaptMkDerivation is to allow attribution removal before passing to the base build helper.

[ShamrockLee]

It's possible to add it as a new feature, which would be great for extendMkDerivation.

Update:

Such a feature would also be surprising if the derived build helper doesn't have ellipses as an edge case.

Most mkDerivation-like build helpers have ellipsis to accept arbitrary arguments and pass them as derivation attributes. It would benefit them to inherit __functionArgs from the base build helper.

Trivial build helpers (defined in pkgs/build-support/trivial-builders/default.nix) usually avoid ellipses and take attributes from derivationArgs instead. Still, such situations would fall into the adaptMkDerivation use cases if support for fixed-point arguments is desired.

I like the idea of __functionArgs inheritance implemented in extendMkDerivation. Do I miss any other edge cases?

[jian-lin]

Currently, the __functionArgs of derived build helpers don't include those from their based build helpers due to the nature of lib.functionArgs.

Could you elaborate on what nature of lib.functionArgs prevents derived build helpers from getting __functionArgs of their base build helpers? I think it is doable if (adaptArgs { }) is changed to something like this:

(lib.setFunctionArgs
  (_: _)
  (lib.functionArgs mkDerivationBase // lib.functionArgs (adaptArgs { })))

It's possible to add it as a new feature ... Such a feature would also be surprising if the derived build helper doesn't have ellipses as an edge case.

I agree. What about adding a new argument to lib.extendMkDerivation and friends to control their behavior? This new argument is an attribute set and currently it has only one attribute: { inheritArgsFromBase ? true }. Note that this attribute set leaves space for future extension.

it might not be a good idea for adaptMkDerivation- derived build helpers to inherit __functionArgs from the base build helper, as the primary goal of adaptMkDerivation is to allow attribution removal before passing to the base build helper.

I do not follow. Why is attribution removal relevant here?

[ShamrockLee]

Currently, the __functionArgs of derived build helpers don't include those from their based build helpers due to the nature of lib.functionArgs.

Could you elaborate on what nature of lib.functionArgs prevents derived build helpers from getting __functionArgs of their base build helpers?

By saying "currently," I mean "before the merge of this PR." Current build helpers take plain attribute sets, and the current way of defining new build helpers is to write new attribute-set-taking functions. The build helpers' current __functionArgs do not include the ones from their base build helpers.

I do not follow. Why is attribution removal relevant here?

We could already define new build helpers taking fixed-point arguments by passing all the user-specified arguments down to the base build helper and exploiting the <pkg>.overrideAttrs of the resulting package. extendMkDerivation is essentially a wrapper around <pkg>.overrideAttrs.

This method only works when defining new build helpers rather than when migrating existing ones because the existing ones only pass part of the set of arguments down the base build helpers. It's easy to pass the arguments from plain attribute sets selectively but challenging when taking fixed-point arguments ((finalAttrs: { })).

We could solve the issue by refactoring the current build helpers and passing each argument properly, which often implies mass rebuilds and long review time. To speed up the adoption of fixed-point arguments support, this PR introduces another function, adaptMkDerivation, as a quick and no-rebuilt workaround for the above issue.

As some of the arguments might be removed before passing to mkDerivationBase when using adaptMkDerivaiton, inheriting the __functionArgs from the mkDerivationBase with adaptMkDerivaiton would be unsuitable. In the long run, we should encourage the migration from adaptBuildHelper to extendBuildHelper for non-trivial build helpers to simplify overriding and eliminate the use of custom overriders (e.g., overridePythonAttrs).

It's possible to add it as a new feature ... Such a feature would also be surprising if the derived build helper doesn't have ellipses as an edge case.

I agree. What about adding a new argument to lib.extendMkDerivation and friends to control their behavior? This new argument is an attribute set and currently it has only one attribute: { inheritArgsFromBase ? true }. Note that this attribute set leaves space for future extension.

Taking an additional attribute set to configure the behavior of extendMkDerivation is a solution. Still, it seems a bit involved always to add an extra { } to make the __functionAttrs prettier.

I'm considering adding the __functionArgs inheritance to extendMkDerivation. People could still choose adaptMkDerivation or the manual <pkg>.overrideAttrs should there be any edge case.

[ShamrockLee]

I added the __functionArgs inheritance to extendMkDerivation. Please take a look.

[jian-lin]

Thanks for the detailed explanation!

I should have been more clear about what I do not follow. I understand the motivation for lib.adaptMkDerivaiton well.

As some of the arguments might be removed before passing to mkDerivationBase when using adaptMkDerivaiton, inheriting the __functionArgs from the mkDerivationBase with adaptMkDerivaiton would be unsuitable.

What I do not quite get is the logic behind this statement. Now after thinking about it for a while, I figure it out. If the base build helper has an argument with the same name as the argument of the derived build helper you want to remove, then not inheriting arguments may be better. However, in other cases, I think it is better to do the inheritance.

[ShamrockLee]

Thank you for explaining. I must elaborate on the relation between attribute removal and its influence on the preference to/not inherit __functionArgs from the base build helper.

You're right. The argument removal doesn't make it surprising to inherit __functionArgs from the base build helper. The inheritance would be unexpected when the developer of the derived build helper wants people to refrain from using specific arguments from the base build helper.

Take buildPythonPackage and buildPythonApplication for example. For historical reasons, the buildPyton* build helpers take checkPhase-related arguments to determine the installCheck behaviors. Renaming the arguments to the installCheckPhase-related ones would make them more correct, but such correctness alone doesn't justify the painful mass renaming. As a result, buildPython* build helpers ask the user to stick to the checkPhase-related arguments and avoid specifying the installCheckPhase-related ones. Assuming that someday stdenv.mkDerivation provides __functionArgs for all its documented attributes, the installCheckPhase-related arguments would also appear in the __functionArgs of buildPythonPackage if adaptMkDerivation does the inheritance.

A counterargument would be that keeping a specific argument inside the function set pattern does not necessarily imply endorsement of such an argument. For instance, buildGoModule keeps buildFlags and buildFlagsArray in its set pattern but explicitly discourages its specification with evaluation warnings.

[ShamrockLee]

Ah, I just forgot the ellipsis. Non-'mkDerivation'- like build helpers— trivial build helpers, fetchers, etc.—usually use derivationArgs instead of ellipsis. They need adaptMkDerivation to remove derivationArgs while taking fixed-point arguments. Whether or not we should add fixed-point arguments support to trivial build helpers is debatable, but it's good to leave some room for such support.

An extra configuration attribute set might be a good idea. If only we have builtins.functionInfo (NixOS/nix#8961 or NixOS/nix#8908) or something like that!

[ShamrockLee]

Would it be desirable to add an extra attribute set to configure the behavior of lib.extendMkDerivation and lib.adaptMkDerivation? This way, we could have lib.adaptMkDerivation { modify = ...; } instead of an additional lib.adaptMkDerivationModified, and lib.adaptMkDerivation { } for the default behavior.

lib.adaptMkDerivation { modify = ...; } stdenv.mkDerivation (finalAttrs:

{ preferLocalBuild ? true
, allowSubstitute ? false
, impassableArg ? (x: false)
, ...
}@args:

{
  inherit preferLocalBuild allowSubstitute;
  # Some other build process
} // removeAttrs args [ "impassableArg" ])

[ShamrockLee]

extendMkDerivation and adaptMkDerivation now take an extra argument set to configure their behaviors. One can leave them empty (adaptMkDerivation { } stdenv.mkDerivation (finalAttrs: args: { ... })) for the default behavior. This change allows customization like adaptMkDerivation { modify = drv: ...; } stdenv.mkDerivation (finalAttrs: args: { ... }) without adding another variant like adaptMkDerivationModified.

I also adopted the relevant format for writing comment block documentation for lib.extendMkDerivation and lib.adaptMkDerivation and formatted all the examples inside fixed-point-arguments.chapter.md with nixfmt.

[jian-lin]

I do not know the motivation and use case of exposing attrsOverlay. Considering composability only, should the exposed overlay take mkDerivationBase into consideration? Something like

Suggested change

	inherit attrsOverlay;
	attrsOverlay = lib.composeExtensions mkDerivationBase.attrsOverlay or { } attrsOverlay;

[ShamrockLee]

I exposed attrsOverlay so that people could experiment with the idea of applying attribute overlays to overrideAttrs proposed by Robert in the review comment:

#234651 (review)

Considering composability, maybe we could change it to attrsOverlays?

{
  attrsOverlays = mkDerivationBase.attrsOverlays or [ ] ++ [ attrsOverlay ];
}

[ShamrockLee]

Addressed.

[ShamrockLee]

After more consideration, I think it would be easier to expose only attrsOverlay and mkDerivationBase. We could add more later if need be.

[jian-lin]

Suggested change

	(
	# Inherit the __functionArgs from the base build helper
	optionalAttrs inheritFunctionArgs (functionArgs mkDerivationBase)
	# Recover the __functionArgs from the derived build helper
	// functionArgs (attrsOverlay { })
	)
	(
	# Recover the __functionArgs from the derived build helper
	functionArgs (attrsOverlay { })
	# Inherit the __functionArgs from the base build helper
	// optionalAttrs inheritFunctionArgs (functionArgs mkDerivationBase)
	)

Note that overlay-a is before overlay-b (the reverse order of adaptMkDerivation).

extendMkDerivation { } (extendMkDerivation { } stdenv.mkDerivation overlay-a) overlay-b

---

Another question: If overlay-a sets one argument foo of functionArgs (overlay-b { }) and foo is not in functionArgs (overlay-a { }), should that argument foo be removed from the result __functionArgs? A similar question also applies to adaptMkDerivation.

[ShamrockLee]

should that argument foo be removed from the result __functionArgs?

Ideally, yes. However, there's no programmatic way to achieve such removal.

[ShamrockLee]

I rebased the branch to resolve merge conflicts, restructured the test.overriding test cases according to (#330434), and improved the documentation.