Raspberry Pi 5 support #260754
Comments
|
"NixOS" proper won't see support for the Raspberry Pi 5. Just like all other ARM platforms, NixOS support depends on the platform's support in upstream mainline projects. We will not repeat the mistake we did with the Raspberry Pi 4 and adding bespoke support for a proprietary ecosystem. It is too much work to support for the resources available to NixOS. Note that this reply applies to any So the roadmap to "NixOS booting trivially on Raspberry Pi 5" is:
Then we can add its U-Boot build to the token pre-baked firmware partition (assuming we still ship it by then) and it should work [just as well as mainline supports it]. An additional side-note: With the super-powers conferred to the end-users of NixOS, nothing stops someone from providing useful configurations, overlays, etc to make it work in the meantime as a workaround. Though as implied, it would be user-supported. |
samueldr
added
the
2.status: wait-for-upstream
|
I think we're all on the same page that it'd be great to have official support but as @samueldr says a lot of work has gone into supporting the Pi (and the oddities of other SOCs). https://github.com/NixOS/nixos-hardware/commits/master/raspberry-pi/4 A nix-community project might be more appropriate this time around so there's a centralized spot to collaborate but it's still distinctly marked unofficial. vision5/riscv support has been going alright without a more central project due to a couple of vocal and active community members + a bit of sponsorship support |
|
I wasn't able to get the Pi 5 to boot with NixOS. The boot loader shows me the following: Wasn't able to get it to boot - no matter what I've tried. |
|
My pi5 needs Nixos :-) |
You need to use patched kernel sources from Raspberry. You also need to use device tree files from Raspberry and merge them using opensource but non-standard I wholeheartedly support the decision not to deal with these oddities officially and support only boards with all drivers & patches & DTSs in the upstream kernel & having standard EFI! But it is NOT that hard to get a new embedded system running in Nix. Not at all. I actually found Nix through embedded systems when I was fed up after many years with I do plan to get Pi5 running along with a simplified disk image generation (lots of unneeded magic stuff surrounding the current |
|
There are actually patches for u-boot for the rpi5, though they're far from complete. I was able to use that to get a very haphazard build of nixos's sd image booting, but it's pretty rough. |
|
I got things sort of working without U-Boot: |
|
NixOS seems to "just work" with https://github.com/worproject/rpi5-uefi, though device support is currently very limited (Ethernet and fan control are the most painful omissions for headless/server use). For some applications, it might already be enough, though. Compared to U-Boot, this has the major benefit of being able to just use the standard aarch64 installer with a mainline kernel. If device tree support gets added to EDK2, it'd hopefully be possible to switch to the vendor kernel post-installation for more complete hardware support, without needing a special Pi-specific installer image. Using the vendor kernel with ACPI might be superior in some senses, but this seems fragile, and I really doubt it'd be considered a supported usecase by the driver developers. |
|
There are a number of reasons this is a bad idea, but installing NixOS using EDK2 and then adding this to the configuration seems to mostly work: {
boot.kernelPackages = (import <nixpkgs-rpi5> {}).linuxPackages_rpi5;
boot.kernelParams = [ "dtb=\\bcm2712-rpi-5-b.dtb" ];
} |
|
rpi5-uefi 0.2 adds device tree support, which seems to work perfectly when using the vendor kernel with UEFI! I'll try to make a flake with the Pi 5 kernel and start writing a wiki page. |
|
I've created a preliminary wiki page with basic installation instructions: https://nixos.wiki/wiki/NixOS_on_ARM/Raspberry_Pi_5 |
|
The above wiki entry is very helpful for getting the PI to boot, but I've had no success booting to X11 or Wayland. While the wiki does list "Graphics drivers are not yet available in NixOS.", this surprises me, given that mesa 23.3.0 has since made it into nixpkgs, which I believe has the drivers for the pi's GPU. If anyone gets graphics working, I'd be curious how :) |
|
@leo60228 I tested rpi5-uefi and you flake, and it seems to be working perfectly! Thanks a lot! I think the only addition in your flake compared to the upstream linux-rpi vendor kernel is the added defconfig line? If so, would you mind submitting a PR so we can just use linuxKernel.kernels.linux_rpi5 (and hopefully be able to use the binary cache)? |
I made that flake before the recent PR updating linux-rpi in Nixpkgs, but this is more or less correct. I was waiting on that PR to be merged before submitting one adding linux_rpi5, I'll try to do that once I'm home tonight if I remember. |
|
Realistically, we should be able to build one rpi kernel for rpi 3, 4, and 5. That's how raspbian works. We can even do the same with u-boot, using u-boot's |
|
Apparently according to https://www.raspberrypi.com/documentation/computers/linux_kernel.html, rpi5 uses a different config so it uses 16K page instead of 4K. |
|
@nbdd0121 hm, seems like it's a tradeoff of performance vs compatibility. For performance, use the rpi5 specific kernel, for compatibility, use the rpi4 one. |
|
I've hacked around and got the installation medium booting from the EFI. Does anyone know if it's possible to install NixOS to the same medium (SD card) as the EFI? The EDK2 docs are elusive. |
|
I used the wiki page @leo60228 created as somewhat of a guide (thanks - you're a star), got my Pi 5 to boot the UEFI from SD, imaged the aarch64 minimal installer to a USB drive and booted that. From there I was indeed able to install to the SD, with NixOS's boot images sharing the same partition as the UEFI loader. Whether that's a good or bad idea I don't know. Not sure what would happen if you somehow booted the installer from the same SD however. I suspect having the partition(s) mounted might hinder those efforts. 🤷 A few notes from my experience which may be useful:
That aforementioned partition table issue I do plan to rectify in due course, but currently that kernel build from @leo60228 is unfortunately an overnight job for my Pi, so I don't have the free time to keep retrying at the moment. If I do get around to trying that then I'll report my findings, good or bad, unless someone beats me to it. |
Try: Then |
|
I'm another who has got it working with rpi5-uefi and @leo60228's flake! The debian UEFI docs where useful in understanding you can extract the bootloader to the same partition as the EFI implementation. For posterity, here are some step-by-step instructions. Format your SD-card, create filesystems, and extract the Raspberry Pi UEFI EDK2 implementation to the boot entry: Flash NixOS arm64 minimial installer to USB drive: On the Pi now, boot into the installer and network it over WiFi (as Ethernet doesn't work). Mount the SD-card: Generate and edit your config with You'll want to set: To install the bootloader to And to add the vendor kernel with e.g.: Then you can install NixOS with Then to boot the vendor kernel, in the UEFI settings switch from ACPI to Device Tree in I also removed NB I couldn't get NixOS to boot automatically even adding the So I need to manually boot from file: Perhaps this is related to using a GPT partition table. I also tried to a cross-compiled installation straight to the SD card from an x86_64 machine which failed due to grub's perl dependency not building.Perhaps cross-compiling with QEMU would work. |
|
This issue has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/install-nixos-on-raspberry-pi-5/38833/2 |
I don't think EFI runtime variables are supported (yet?), so you should set
You need to add |
|
Thanks for all the useful conversation had here and resources put together! I was able to get NixOS installed with relative ease on my Pi 5 yesterday, which has now replaced my previous NixOS Pi 4 as a DHCP, DNS, WireGuard, and nginx server for my network and has been working great, much more performant ✨ I figured I'd share a little bit of my experience here in case it helps others. Then proceeded with the mounting. Generate the config. Edit the config to what should roughly be a minimal working system: configuration.nix{ config, lib, pkgs, ... }:
{
imports = [ ./hardware-configuration.nix ];
boot.loader.systemd-boot.enable = true;
boot.loader.efi.canTouchEfiVariables = false;
networking.hostName = "rpi5"; # Define your hostname.
networking.wireless.iwd = {
enable = true;
settings.General.EnableNetworkConfiguration = true;
};
time.timeZone = "Europe/London";
environment.systemPackages = with pkgs; [
vim
curl
git
];
# just for access after install to continue provisioning
services.openssh.enable = true;
services.openssh.settings.PermitRootLogin = "yes";
networking.firewall.enable = false;
system.stateVersion = "23.11";
}hardware-configuration.nix{ config, lib, pkgs, modulesPath, ... }:
{
imports =
[
(modulesPath + "/installer/scan/not-detected.nix")
];
boot.initrd.availableKernelModules = [ "usbhid" "usb_storage" ];
boot.initrd.kernelModules = [ ];
boot.kernelModules = [ ];
boot.kernelPackages = (import (builtins.fetchTarball https://gitlab.com/vriska/nix-rpi5/-/archive/main.tar.gz)).legacyPackages.aarch64-linux.linuxPackages_rpi5;
boot.extraModulePackages = [ ];
fileSystems."/" =
{
device = "/dev/disk/by-uuid/ecb26648-686e-403c-a415-406ac554653d";
fsType = "ext4";
};
fileSystems."/boot" =
{
device = "/dev/disk/by-uuid/0CAE-25FE";
fsType = "vfat";
};
swapDevices = [ ];
networking.useDHCP = lib.mkDefault true;
nixpkgs.hostPlatform = lib.mkDefault "aarch64-linux";
}Made sure to set Before continuing with the install, here's what others may find particularly useful: the Pi is going to take a loooooong time to compile the kernel. If, like me, you happen to be using NixOS on another, more ample aarch64 system (I'm running NixOS with Asahi on an M2 MacBook Air), you can just build your Pi's system closure on that and then copy it to the installer's Nix store:
(if you have problems with impurity because of flakes, either pass
I appreciate that's a bit of a 🧠 dump, so please feel free to ask for any clarification on the above steps. One other curiosity I had before I did the install was if ethernet worked once booted to NixOS with the vendor kernel and Device Tree/System Table mode. It wasn't clear to me based on comments/info available if it would. For anyone wondering: it does :) |
|
Big thanks for the kernel packages and guides.
For anyone else who's curious, I ran the experiment to put a number on it. Haven't gone through with the installation yet, but building from a Pi 5 4GB running debian on an nvme drive, the result was: $ nix build -L .#nixosConfigurations.....config.system.build.toplevel
# ...
took 2h9m50sIt's long, but it's not waiting for hydra long. ;) |
|
See for more on the raspberry pi bootloader #241534 |
I have not updated my config to build a Pi 5 image. I said that it can be made (as in modified/updated) to build a Pi 5 image. The installation is split into two steps because unfortunately the systemd-boot's (which is what I'm using at the moment) install script requires running on target's architecture. It could be rewritten to be generic and or I could also try using grub.
I've seen people doing nix store and closure copying magic manually without I want to simplify it further by removing I'm also not interested at all at using uboot (or for f'sake native raspberry bootloader which doesn't even support boot entries so whole NixOS concept of booting to a previous/different configuration goes out of window). qcow2 is inconsequential. Anything compressed is OK. Since the VM runs are derivations and that image is their input/output cached in a nix store, you don't want it to be uncompressed. Same goes for the final image. It's zstd compressed in order not to take precious disk space. |
|
Hello! I have a probably very ridiculous question, but I don't have the knowledge to figure out the answer by myself. Why is it not possible to create nix definition that takes the kernel and bootloader used in Raspbian, and just uses it to boot rpi5? I mean this code here: Also thanks a lot for the very informative thread. EDIT: So I see we are already using the firmware here. But we are not pulling the kernel right? EDIT2: We are also getting the linux from RPI here. |
Of course it is possible. Why wouldn't it be possible? |
But then, and this is probably coming from my ignorance, I don't understand what is the big problem in supporting rpi5? I mean, it's just a matter of specifying the kernel from Raspbian no? Ugly pseudo code incomming: boot.kernelPackages = pkgs.linuxPackagesGetKernelFromRaspbian;I don't have an rpi5 to try this out. In fact, I am using this thread to make a decision on if I should buy an Rpi5 or not :D |
|
You've already got |
I see.. Hopefully it will also work on 24.05 right? But if we could all along pull the kernel from raspbian (which works on all rpis), what was the problem with supporting rpi5? Sorry for the torrent of noob questions. |
I'll try my best to provide some more context on why, generally speaking, your ideas for an external kernel may not be considered a conventional approach in Nix :) Pulling in a pre-compiled kernel from an external source who's build inputs, environment, configuration, and instructions for use in a system that's (likely) trying its best to be declarative, reproducible, and hermetic is a bit of an anti-pattern. As others have suggested, I'm sure you could get it to work, no problem. But it's not really in keeping with Nix ideals. Now, that's just for use in one's personal configuration. If I'm understanding correctly, you're also suggesting to potentially have that inherited in nixpkgs/NixOS? Perhaps I'm misreading it, but on top of the above reasons (which become far more "serious" when considering nixpkgs/NixOS): even if the good folks here maintaining NixOS were to decide to inherit the build process for the Raspbian kernel, maintaining platform specific kernels has proven an absolute nightmare in the past. Raspbian is a project focused solely on providing a distribution for one set of boards. NixOS is intended to be a general purpose distribution, capable of running on many systems. Hopefully that clears some things up? :) Essentially, there's a history in NixOS of a very-loosely similar path being walked and it was pretty awful from a maintainability perspective. |
|
Hi @cmacrae! Thanks a lot for the detailed response. It really helps beginners like me. I also understand now the point that everyone was mentioning: The mainline Linux Kernel does not yet support rpi5. This is what I was not understanding. Once again, thanks :) |
|
I researched a bit about the boot process on the Pi 5 in order to better understand what we are adapting here. I summarized my understanding at https://wiki.nixos.org/wiki/NixOS_on_ARM/Raspberry_Pi_5. Please correct and extend what is wrong or missing. |
I was also fighting the GPU issue, figured it out today! We're forgetting to include the raspberrypi overlay. I threw some more instructions up here https://github.com/celesrenata/nix-flakes/tree/rpi5/README.md Hopefully they help somebody! |
|
I knew I was forgetting a step when I added it to the wiki, but couldn't think of what it was.... oops. |
|
I just found out that U-Boot in its v2024.04 release got basic pi 5 support (e.g. booting from SD but not USB,NVMe): Is anyone able to update (or link to the MR) # flake.nix
{
# kernel suitable for pi 5
inputs.nixos-hardware.url = "github:malteneuss/nixos-hardware/raspberry-pi-5";
inputs.nixpkgs-unstable.url = "nixpkgs/nixos-unstable";
outputs =
inputs@{ self
, nixos-hardware
, nixpkgs-unstable
, ...
}:
nixosConfigurations.pi5 = nixpkgs-unstable.lib.nixosSystem {
modules = [
my-configuration.nix
({ pkgs, modulesPath, ... }: {
nixpkgs = {
localSystem = {
system = "x86_64-linux";
};
crossSystem = { # <--------------cross-compile on Intel/AMD machine. Emulated native compilation with binfmt too slow.
system = "aarch64-linux";
};
# workaround for breaking linux kernel changes https://github.com/NixOS/nixpkgs/issues/154163#issuecomment-1350599022
overlays = [ # <-------------------------------
(final: super: {
makeModulesClosure = x: super.makeModulesClosure (x // { allowMissing = true; });
})
];
}; # nixpkgs end
imports = [
nixos-hardware.nixosModules.raspberry-pi-5
"${toString modulesPath}/installer/sd-card/sd-image-aarch64.nix"
];
})
];
};
};which could then be build into an SD card image using the regular nix build .\#nixosConfigurations.pi5.config.system.build.sdImageedit: the new u-boot is already in nixpkgs-unstable: nixpkgs/pkgs/misc/uboot/default.nix Line 31 in 8f9b659 |
|
Using |
|
@leo60228 These are two good setup variations that should be documented somewhere, probably on the Wiki page (i would love a setup where i don't need to fiddle around and install Nix on the Pi to build Nix stuff; just generate a ready NixOS image, Since u-boot in nixpkgs has basic pi 5 support, what can we do now to create a "final" sd card image? boot.loader.generic-extlinux-compatible.enable = true;
boot.loader.systemd-boot.enable = false;
boot.loader.efi.canTouchEfiVariables = false;should be sufficient. |
@malteneuss You may have a look at this project: https://github.com/tstat/raspberry-pi-nix. Here is for https://github.com/tstat/raspberry-pi-nix/blob/master/sd-image/default.nix if you just want to know how sd-image with uboot is generated. |
|
Just wanted to report back that @nvmd suggestion of https://github.com/tstat/raspberry-pi-nix so far has the simplest works-out-of-the-box experience i've seen so far; just use the example there. It seems to package the official firmware and kernels for Pi 5 and works without u-boot, as well as soon with u-boot nix-community/raspberry-pi-nix#13. However, it can't be cross-compiled (yet), but setting up the Pi as a remote native builder is easier than i thought:
I've simplified the guide on https://wiki.nixos.org/wiki/Distributed_build a bit on the details for that. fyi: The whole build for a final, pre-configured NixOS sd card image takes ~3h on the Pi 5 8GB. You can then |
hetzner has very reasonably priced arm servers. I set one up as a a builder using nixos-anywhere. You can use the lowest tier and scale it up anytime you need to perform a kernel build. Using the 16 core instance it takes about 40 minutes! |
|
So, I ran into new problems in 24.05 and the RPi5, for some reason one day, as I rolled up changes, my wifi drivers went poof! No matter what I couldn't get them back. So I decided to try to start my journey to upgrade out of the problem. I've taken a few stabs at it over the weeks thanks to @leo60228's kernel base. I ended up with this working when married with the following overlay https://github.com/raspberrypi/firmware/tree/f1266637375cd7880946dbaceda871f3f308d901/boot/overlays We now have access to 6.6.35 main! I noticed that stable 6.6.28 and 6.6.31 do not build for us, however, this does! You will need to update your kernel packages with.... (https://github.com/celesrenata/nix-flakes/blob/rpi5/nixberry/boot.nix) kernelPackages = let
linux_rpi5_pkg = { stdenv, lib, buildPackages, fetchFromGitHub, perl, buildLinux, rpiVersion, ... } @ args:
let
# NOTE: raspberrypifw & raspberryPiWirelessFirmware should be updated with this
modDirVersion = "6.6.35";
tag = "rpi-6.6.y";
in
lib.overrideDerivation (buildLinux (args // {
version = "${modDirVersion}-${tag}";
inherit modDirVersion;
src = fetchFromGitHub {
owner = "raspberrypi";
repo = "linux";
rev = "c1432b4bae5b6582f4d32ba381459f33c34d1424";
hash = "sha256-pzjgCWG9FhMU3LCZnvz5N4jYfaaJQDT6Pv5lD/3zsm4=";
};
defconfig = {
"1" = "bcmrpi_defconfig";
"2" = "bcm2709_defconfig";
"3" = if stdenv.hostPlatform.isAarch64 then "bcmrpi3_defconfig" else "bcm2709_defconfig";
"4" = "bcm2711_defconfig";
"5" = "bcm2712_defconfig";
}.${toString rpiVersion};
features = {
efiBootStub = false;
} // (args.features or {});
# Left over from a previous struggle and can be deleted
makeFlags = [
"CONFIG_CEPH_LIB=n"
"CONFIG_CEPH_FS=n"
"CONFIG_CEPH_FSCACHE=n"
"CONFIG_CEPH_FS_POSIX_ACL=n"
"CONFIG_EROFS_FS_XATTR=n"
"CONFIG_EROFS_FS_POSIX_ACL=n"
"CONFIG_EROFS_FS_SECURITY=n"
"CONFIG_EROFS_FS_ZIP=n"
"CONFIG_SND_SOC_SIMPLE_MUX=n"
];
# /Delete
extraMeta = if (rpiVersion < 3) then {
platforms = with lib.platforms; arm;
hydraPlatforms = [];
} else {
platforms = with lib.platforms; arm ++ aarch64;
hydraPlatforms = [ "aarch64-linux" ];
};
} // (args.argsOverride or {}))) (oldAttrs: {
postConfigure = ''
# The v7 defconfig has this set to '-v7' which screws up our modDirVersion.
sed -i $buildRoot/.config -e 's/^CONFIG_LOCALVERSION=.*/CONFIG_LOCALVERSION=""/'
sed -i $buildRoot/include/config/auto.conf -e 's/^CONFIG_LOCALVERSION=.*/CONFIG_LOCALVERSION=""/'
'';
# Make copies of the DTBs named after the upstream names so that U-Boot finds them.
# This is ugly as heck, but I don't know a better solution so far.
postFixup = ''
dtbDir=${if stdenv.isAarch64 then "$out/dtbs/broadcom" else "$out/dtbs"}
rm $dtbDir/bcm283*.dtb
copyDTB() {
cp -v "$dtbDir/$1" "$dtbDir/$2"
}
'' + lib.optionalString (lib.elem stdenv.hostPlatform.system ["armv6l-linux"]) ''
copyDTB bcm2708-rpi-zero-w.dtb bcm2835-rpi-zero.dtb
copyDTB bcm2708-rpi-zero-w.dtb bcm2835-rpi-zero-w.dtb
copyDTB bcm2708-rpi-b.dtb bcm2835-rpi-a.dtb
copyDTB bcm2708-rpi-b.dtb bcm2835-rpi-b.dtb
copyDTB bcm2708-rpi-b.dtb bcm2835-rpi-b-rev2.dtb
copyDTB bcm2708-rpi-b-plus.dtb bcm2835-rpi-a-plus.dtb
copyDTB bcm2708-rpi-b-plus.dtb bcm2835-rpi-b-plus.dtb
copyDTB bcm2708-rpi-b-plus.dtb bcm2835-rpi-zero.dtb
copyDTB bcm2708-rpi-cm.dtb bcm2835-rpi-cm.dtb
'' + lib.optionalString (lib.elem stdenv.hostPlatform.system ["armv7l-linux"]) ''
copyDTB bcm2709-rpi-2-b.dtb bcm2836-rpi-2-b.dtb
'' + lib.optionalString (lib.elem stdenv.hostPlatform.system ["armv7l-linux" "aarch64-linux"]) ''
copyDTB bcm2710-rpi-zero-2.dtb bcm2837-rpi-zero-2.dtb
copyDTB bcm2710-rpi-3-b.dtb bcm2837-rpi-3-b.dtb
copyDTB bcm2710-rpi-3-b-plus.dtb bcm2837-rpi-3-a-plus.dtb
copyDTB bcm2710-rpi-3-b-plus.dtb bcm2837-rpi-3-b-plus.dtb
copyDTB bcm2710-rpi-cm3.dtb bcm2837-rpi-cm3.dtb
copyDTB bcm2711-rpi-4-b.dtb bcm2838-rpi-4-b.dtb
'';
} // (args.argsOverride or {}));
linux_rpi5 = pkgs.callPackage linux_rpi5_pkg{
kernelPatches = with pkgs.kernelPatches; [
bridge_stp_helper
request_key_helper
];
rpiVersion = 5;
};
in
pkgs.recurseIntoAttrs (pkgs.linuxPackagesFor linux_rpi5);I am currently sitting happily on wifi on Hyprland again on my RPi5. |
|
It's very sad to hear that this issue has been closed and not planned :( I run NixOS successfully on 4 rpi 4, and I have started upgrading to rpi 5 slowly. It's a pity that the interest is not there, and I wish I could help somehow as I know that I am not alone! I know there are difficulties, practical and philosophical. The user base of these board is massive, and I believe I am not the only one eyeing it! Is this something that will change in the future at some point? |
I meant that I accidentaly referenced this issue link on a PR I did 😅. Thanks to the awesome guides in this issue, I managed to install NixOS on a Raspberry PI 5, it's even running Sway (wayland). I'm really happy with it and appreciate all the hard work you guys did! |
|
What do you mean by accidentally opened the issue? The issue is opened @cor ~9 month ago. |
just to make sure it is planned then but u referencing it somehow closed it? or am i missing something? |
|
I'm a little confused - prior to @redyf closing as not planned 3 days ago, I believe this issue was still set to being open as I check it regularly? I see no reference to it being closed prior either. So why is it now closed? |
|
This was a HUGE misunderstanding on my part, I apologize for anyone I affected by closing the issue. The concept of issues just wasn't clear enough for me and I thought I could just clear the reference I did to the commit. I had no clue I could actually close the issue that has been opened by other users. In summary, I'm dumb 😅 |
and others
Has anyone been able to run NixOS on the newly released Raspberry Pi 5? It's not included yet in the wiki's compatibility matrix.
The text was updated successfully, but these errors were encountered: