Can't override python3 in an overlay

[kjeremy]

I can't seem to override python3 anymore. Previously I had an overlay that did the following:

(self: super:
python3 = super.python3.override {
  self = self.python3;
  # enable optimization etc.
  packageOverrides = ...
};)

to override some packages for cross compilation. This pattern now fails. I have a reproducer here: https://github.com/kjeremy/nix-ros-overlay-python-fail.

[jkolb@nixos:~/nix-ros-overlay-test]$ nix develop .#example
error:
       … while calling the 'derivationStrict' builtin
         at <nix/derivation-internal.nix>:9:12:
            8|
            9|   strict = derivationStrict drvAttrs;
             |            ^
           10|

       … while evaluating derivation 'nix-shell'
         whose name attribute is located at /nix/store/snp5f95xhkz89kdw9703kana3m6xfd0a-source/pkgs/stdenv/generic/make-derivation.nix:352:7

       … while evaluating attribute 'nativeBuildInputs' of derivation 'nix-shell'
         at /nix/store/snp5f95xhkz89kdw9703kana3m6xfd0a-source/pkgs/stdenv/generic/make-derivation.nix:396:7:
          395|       depsBuildBuild              = elemAt (elemAt dependencies 0) 0;
          396|       nativeBuildInputs           = elemAt (elemAt dependencies 0) 1;
             |       ^
          397|       depsBuildTarget             = elemAt (elemAt dependencies 0) 2;

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: attribute 'catkin-pkg' missing
       at /nix/store/zv36pjyxlsg6il3k6zhc1wdmgclwqqw9-source/distros/noetic/catkin/default.nix:18:41:
           17|   checkInputs = [ python3Packages.mock python3Packages.nose ];
           18|   propagatedBuildInputs = [ cmake gtest python3Packages.catkin-pkg python3Packages.empy python3Packages.nose python3Packages.setuptools ];
             |                                         ^
           19|   nativeBuildInputs = [ cmake python3Packages.setuptools ];

As an aside (hint?) in my actual project it complains about empy_3 missing.

[kjeremy]

I thought the following would work too but it fails to take my modifications into account.

python3 = super.python3 // {
  pkgs = super.python3.pkgs.overrideScope(pyFinal: pyPrev: {
    mypkg = pyPrev.mypkg.overrideAttrs ....
  });
};

EDIT: In this case my modifications are available under python3.pkgs but not python3Packages.

[kjeremy]

Okay I can work around the package issue by using pythonPackagesExtensions like the following in an overlay:

pythonPackagesExtensions = super.pythonPackagesExtensions ++ [
  (pyFinal: pyPrev: {
    pykdl = pyPrev.pykdl.overrideAttrs({ cmakeFlags ? [], ... }: {
      cmakeFlags = cmakeFlags ++ [ "-DPYTHON_EXECUTABLE=${lib.getExe self.python3.pythonOnBuildForHost}" "-DWHATEVER=1" ];
    });
  })
];

This allows my version of pykdl to be present in both python3.pkgs and python3Packages. However I still cannot override python3 in the overlay without getting the error about empy_3 missing.

[kjeremy]

@lopsided98 do you have any suggestions? This is a blocker for me.

[Pleune]

I had this same issue, and ended up just copying the pythonOverridesFor function into my flake and applying it there too. Not the best solution, but the function is short enough I did it anyways.

# from distros/distro-overlay.nix
pythonOverridesFor = with self.lib; prevPython: prevPython // {
  pkgs = prevPython.pkgs.overrideScope (pyFinal: pyPrev: {
    wxPython = pyFinal.wxPython_4_2;
    empy = pyFinal.empy_3;
};

then apply with this in your overlay.

python3 = pythonOverridesFor super.python3;

[kjeremy]

Hm. Then you add your package overrides in the implementation of pythonOverridesFor?

Edit: this approach doesn't seem to work for me. I'm still missing empy_3.

[lopsided98]

I switched to pythonPackagesExtensions; does this fix your issue?