Skip to content

Latest commit

 

History

History
510 lines (388 loc) · 21.7 KB

DEVELOPER.md

File metadata and controls

510 lines (388 loc) · 21.7 KB
Raw

winesapOS Developer Guide

Architecture

Partitions

Number Label File System Size Description
1 None None 2 MB BIOS boot backwards compatibility.
2 wos-drive exFAT 16 GB Cross-platform flash drive storage.
3 WOS-EFI FAT32 500 MB UEFI boot firmware.
4 winesapos-boot ext4 1 GB GRUB boot loader and Linux kernel.
5 (Performance) winesapos-root Btrfs 100% The root and home file systems.
5 (Secure) winesapos-luks LUKS 100% The encrypted root and home file systems.

Drivers

Mac

These drivers are provided for better compatibility with the lastest Macs with Intel processors:

  • apple-bce = T2 driver required for storage, mouse, keyboard, and audio support. We provide a git repository that syncs up both the t2linux and macrosfad forks. It provides the newer kernel compatibility from t2linux and also a DKMS module from macrosfad for easily installing the kernel driver.
  • macbook12-spi-driver-dkms = MacBook Pro Touch Bar driver. The package is installed from the AUR.
  • snd_hda_macbookpro = Sound driver. This requires the apple-bce driver to work on some Macs. We provide a git repository that modifies the installation script to install for all Linux kernels found on the system instead of just the running Linux kernel.

Files

These are a list of custom files and script that we install as part of winesapOS:

  • /etc/pacman.conf = On SteamOS builds, this provides the correct order of enabled repositories. [jupiter] and [holo] come first and have package signatures disabled (Valve does not provide any). Then the Arch Linux repositories.
    • Source: files/etc-pacman.conf_steamos
  • /etc/snapper/configs/{root,home} = The Snapper configuration for Btrfs backups.
    • Source: files/etc-snapper-configs-root
  • /etc/systemd/user/winesapos-mute.service = A user (not system) service for muting all audio. This is required for some newer Macs that have in-development hardware drivers that are extremely loud by default.
    • Source: files/winesapos-mute.service
  • /usr/local/bin/winesapos-mute.sh = The script for the winesapos-mute.service.
    • Source: scripts/winesapos-mute.sh
  • /etc/systemd/system/pacman-mirrors.service = On Manjaro builds, this provides a service to find and configure the fastest mirrors for Pacman. This is not needed on Arch Linux builds as it has a Reflector service that comes with a service file. It is also not needed on SteamOS builds as Valve provides a CDN for their single mirror.
    • Source: files/pacman-mirrors.service
  • /etc/systemd/system/winesapos-resize-root-file-system.service = A service that runs a script to resize the root file system upon first boot.
    • Source: winesapos-resize-root-file-system.service
  • /etc/systemd/system/winesapos-touch-bar-usbmuxd-fix.service = A workaround for MacBook Pros with a Touch Bar. This will allow iOS devices to connect on Linux again. This service will show an error during boot if winesapOS boots on a system that is not a Mac with a Touch Bar.
    • Source: files/winesapos-touch-bar-usbmuxd-fix.service
  • /usr/local/bin/winesapos-touch-bar-usbmuxd-fix.sh = The script used for the winesapos-touch-bar-usbmuxd-fix.service.
    • Source: files/winesapos-touch-bar-usbmuxd-fix.sh
  • /usr/share/libalpm/hooks/steamdeck-kde-presets.hook = A Pacman hook that is triggered when the steamdeck-kde-presets package is installed or updated. This will delete a global autostart Steam desktop shortcut as our users may not want Steam to start immediately after login.
    • Source: files/steamdeck-kde-presets.hook
  • /home/winesap/.winesapos/winesapos-setup.desktop = A desktop shortcut for the winesapOS First-Time Setup wizard.
    • Source: files/winesapos-setup.desktop
  • /home/winesap/.winesapos/winesapos-upgrade.desktop = A desktop shortcut for the winesapOS Upgrade wizard.
    • Source: files/winesapos-upgrade.desktop
  • /usr/local/bin/winesapos-resize-root-file-system.sh = The script used for the winesapos-resize-root-file-system.service.
    • Source: scripts/winesapos-resize-root-file-system.sh
  • /home/winesap/.winesapos/winesapos-setup.sh = The script used for the winesapOS First-Time Setup wizard.
    • Source: scripts/winesapos-setup.sh
  • /home/winesap/.winesapos/winesapos_logo_icon.png = The winesapOS logo as a 96x96 icon for the winesapOS First-Time Setup and winesapOS Upgrade desktop shortcuts.
    • Source: files/winesapos_logo_icon.png
  • /home/winesap/.winesapos/winesapos-upgrade-remote-stable.sh = The script used for the winesapOS Upgrade wizard. It pulls the latest upgrade script from the "stable" branch of winesapOS.
    • Source: scripts/winesapos-upgrade-remote-stable.sh
  • /home/winesap/Desktop/README.txt = A read me file on the desktop that links to the GitHub page of winesapOS, explains the winesapOS wizards, and lists out all application shortcuts found on the desktop and what they are for.
    • Source: created as part of scripts/winesapos-install.sh

Build

Download the Installer

Depending on which Arch Linux distribution you want to build, download the related installer. Both Arch Linux and Manjaro provide ISOs for a live CD environment. The Steam Deck recovery image is a block device so it needs to be configured differently in a virtual machine before installing winesapOS.

Create Virtual Machine

A virtual machine is used to build winesapOS in a safe and isolated manner. The disks on the hypervisor will not be touched. It is assumed that QEMU/KVM will be used although other hypervisors can be used.

Requirements:

  • UEFI boot
  • 2 vCPUs
  • 4 GB RAM
  • 28 GiB storage (to fit on a 32 GB flash drive)

CLI

  • Create the virtual storage device.

    • All:

      sudo qemu-img create -f raw -o size=28G /var/lib/libvirt/images/winesapos.img

  • Create the virtual machine to use for installing winesapOS.

    • Arch Linux and Manjaro use an installer ISO image.

      sudo virt-install --name winesapos --boot uefi --vcpus 2 --memory 4096 --disk path=/var/lib/libvirt/images/winesapos.img,bus=virtio,cache=none --cdrom=/var/lib/libvirt/images/<INSTALLER_ISO>

    • SteamOS 3 uses a recovery image.

      sudo virt-install --name winesapos --boot uefi --vcpus 2 --memory 4096 --disk path=/var/lib/libvirt/images/steamdeck-recovery-1.img,bus=virtio,cache=none --disk path=/var/lib/libvirt/images/winesapos.img,bus=virtio,cache=none

GUI

Arch Linux and Manjaro:

  1. Virtual Machine Manager (virt-manager)
  2. File
  3. New Virtual Machine
  4. Local install media (ISO image or CDROM)
  5. Forward
  6. Choose ISO or CDROM install media: <INSTALLER_ISO>
  7. Forward
  8. Memory: 4096
  9. CPUs: 2
  10. Forward
  11. Enable storage for this virtual machine: yes
  12. Create a disk image for the virtual machine: 24.0 GiB
  13. Forward
  14. Name: winesapOS
  15. Customize configuration before install: yes
  16. Finish
  17. Overview
  18. Firmware: UEFI x86_64: /usr/share/edk2-ovmf/x64/OVMF_CODE.secboot.fd
  19. Apply
  20. Begin Installation

SteamOS 3:

  1. Virtual Machine Manager (virt-manager)
  2. File
  3. New Virtual Machine
  4. Import existing disk image
  5. Provide the existing storage path: steamdeck-recovery-1.img
  6. Choose the operating system you are installing: Arch Linux (archlinux)
  7. Forward
  8. Memory: 4096
  9. CPUs: 2
  10. Foward
  11. Name: winesapOS
  12. Customize configuration before install: yes
  13. Finish
  14. Overview
  15. Firmware: UEFI x86_64: /usr/share/edk2-ovmf/x64/OVMF_CODE.secboot.fd
  16. Apply
  17. Add Hardware
  18. Storage
  19. Create a disk image for the virtual machine: 24.0 GiB
  20. Finish
  21. Begin Installation

Environment Variables

For specialized builds, use environment variables to modify the installation settings.

$ export <KEY>=<VALUE>
Key Values Performance Value (Default) Secure Value Description
WINESAPOS_DEBUG_INSTALL true or false true true Use set -x for debug shell logging during the installation.
WINESAPOS_DEBUG_TESTS true or false false false Use set -x for debug shell logging during the tests.
WINESAPOS_BUILD_IN_VM_ONLY true or false true true If the build should fail and exit if it is not in a virtual machine. Set to false for a bare-metal installation.
WINESAPOS_CREATE_DEVICE true or false false false If the build should create and use an image file instead of using an existing block device.
WINESAPOS_DEVICE vda vda If WINESAPOS_CREATE=false, then use the existing /dev/${WINESAPOS_DEVICE} block device to install winesapOS onto.
WINESAPOS_INSTALL_DIR /winesapos /winesapos The chroot directory where winesapOS will be installed into.
WINESAPOS_DISTRO arch, manjaro, or steamos steamos steamos The Linux distribution to install with.
WINESAPOS_DE cinnamon or plasma plasma plasma The desktop environment to install.
WINESAPOS_ENCRYPT true or false false true If the root partition should be encrypted with LUKS.
WINESAPOS_ENCRYPT_PASSWORD password password The default password for the encrypted root partition.
WINESAPOS_LOCALE en_US.UTF-8 UTF-8 en_US.UTF-8 UTF-8 The locale to use for /etc/locale.gen.
WINESAPOS_APPARMOR true or false false true If Apparmor should be installed and enabled.
WINESAPOS_PASSWD_EXPIRE true or false false true If the root and winesap user passwords will be forced to be changed after first login.
WINESAPOS_SUDO_NO_PASSWORD true or false true false If the user can run sudo without entering a password.
WINESAPOS_FIREWALL true or false false true If a firewall (firewalld) will be installed.
WINESAPOS_CPU_MITIGATIONS true or false false true If processor mitigations should be enabled in the Linux kernel.
WINESAPOS_DISABLE_KERNEL_UPDATES true or false true false If the Linux kernels should be excluded from being upgraded by Pacman.
WINESAPOS_DISABLE_KWALLET true or false true false If Kwallet should be enabled for securing various passwords.
WINESAPOS_ENABLE_KLIPPER true or false true false If Klipper should be disabled (as much as it can be) for storing copied text.

Install winesapOS

Once the virtual machine is running, a distribution of Arch Linux for winesapOS can be installed. An automated script is provided to fully install the operating system. This script will only work in a virtual machine. Clone the entire project repository. This will provide additional files and scripts that will be copied into the virtual machine image.

SteamOS 3 requires making the root file system writable, setting up Pacman keyrings for Arch Linux, and then installing the required zsh dependency.

$ sudo steamos-readonly disable
$ sudo pacman -S -y
$ sudo pacman-key --init
$ sudo pacman-key --populate archlinux
$ sudo pacman -S zsh

Arch Linux requires installing the required git dependency.

$ sudo pacman -S -y
$ sudo pacman -S git

Clone the GitHub repository:

$ git clone https://github.com/lukeshortcloud/winesapos.git
$ cd winesapos/scripts/

Before running the installation script, optionally set environment variables to configure the build. Use sudo -E to load the environment variables.

  • Performance-focused image build:

    • Arch Linux and SteamOS 3 hybrid (default):

      # export WINESAPOS_DISTRO=steamos
# ./winesapos-install.sh

    • Arch Linux:

      # export WINESAPOS_DISTRO=arch
# ./winesapos-install.sh

    • Manjaro and SteamOS 3 hybrid:

      $ export WINESAPOS_DISTRO=steamos
$ sudo -E ./winesapos-install.sh

    • Manjaro:

      $ export WINESAPOS_DISTRO=manjaro
$ sudo -E ./winesapos-install.sh

    • SteamOS 3:

      $ export WINESAPOS_DEVICE=vdb
$ sudo -E ./winesapos-install.sh

  • Security-focused image build requires first sourcing the environment variables:

    • Arch Linux and SteamOS 3 hybrid:

      # export WINESAPOS_DISTRO=steamos
# . ./winesapos-env-secure.sh
# ./winesapos-install.sh

    • Arch Linux:

      # export WINESAPOS_DISTRO=arch
# . ./winesapos-env-secure.sh
# ./winesapos-install.sh

    • Manjaro and SteamOS 3 hybrid:

      $ export WINESAPOS_DISTRO=manjaro
$ . ./winesapos-env-secure.sh
$ sudo -E ./winesapos-install.sh

    • Manjaro:

      $ export WINESAPOS_DISTRO=manjaro
$ . ./winesapos-env-secure.sh
$ sudo -E ./winesapos-install.sh

    • SteamOS 3:

      $ export WINESAPOS_DEVICE=vdb
$ . ./winesapos-env-secure.sh
$ sudo -E ./winesapos-install.sh

When complete, run the automated tests and then shutdown the virtual machine (do NOT restart). The image can then be cleaned up and used for manual testing on an external storage device.

Tests

Matrix

These are all of the scenarioes that need to be tested and working before a release.

OS Performance Secure Plasma Cinnamon
SteamOS x x x x
Arch Linux x x x x
Manjaro x x x x

Automatic

Run the tests to ensure that everything was setup correctly. These are automatically ran and logged as part of the install script. The tests must be run with the ZSH shell (not BASH).

$ sudo zsh ./winesapos-tests.sh

Manual

On the hypervisor, clean up the virtual machine image. This will ensure that the image will generated unique values for additional security and stability. The customize operation is disabled because the operation will set a new machine-id which is not what we want. Our image already has a blank /etc/machine-id file which will be automatically re-generated on first boot.

$ sudo virt-sysprep --operations defaults,-customize -a /var/lib/libvirt/images/winesapos.img

Optionally flash the image onto an external storage device for testing hardware. Otherwise, run manual tests using a virtual machine.

$ sudo dd if=/var/lib/libvirt/images/winesapos.img of=/dev/<DEVICE>

Manual tests:

  • Package Managers
    • Discover
      • Install a Pacman package: apache
      • Install a Flatpak package: org.gnome.BreakTimer
    • Pamac
      • Install a Pacman package: nginx
      • Install a Flatpak package: org.gabmus.gfeeds
      • Install an AUR package: cmatrix-git
      • Install a Snap package: lxd

Workflows

Adding Applications

If adding a new application to winesapOS, these are all of the places it needs to be updated:

  • README.md needs to mention that application under the "Usability" or "Gaming support" sections under the "Features" header.
  • CHANGELOG.md needs to mention that the application has has been Added, Changed, or Removed.
  • src/winesapos-install.sh
    • The installer creates a file at "/home/winesap/Desktop/README.txt" that lists every GUI applications.
    • The installer creates shortcut files for GUI applications.
  • src/winesapos-tests.sh needs updated tests to at least check for the existence of the package and desktop shortcut (if applicable).

Importing SteamOS 3 Source Code

SteamOS 3 source code is hosted in an internal GitLab repository at Valve. As a workaround, we can import the git repository from source Pacman packages and use them for building modified applications. The most notable package we need to modify is Mesa to add in Intel OpenGL driver support.

  • Find, download, and extract a source package from either the holo or jupiter SteamOS 3 Pacman repository.

    tar -x -v -f <PACKAGE_NAME>-<PACKAGE_VERSION>.src.tar.gz

  • Notice how there is a PKGBUILD file that can be modified and uploaded to the Arch Linux User Repository (AUR).

    less <PACKAGE_NAME>/PKGBUILD

  • Convert the bare git repository into a regular git repository.

    cd <PACKAGE_NAME>/archlinux-<PACKAGE_NAME>/
mkdir .git
mv branches ./git/
mv config ./git/
mv description ./git/
mv HEAD ./git/
mv hooks ./git/
mv info ./git/
mv objects ./git/
mv packed-refs ./git/
mv refs ./git/
git config --local --bool core.bare false
git reset --hard

  • Add a new remote and then push the entire git repository to it.

    git remote add winesapos [email protected]:<GIT_USER>/<GIT_REPOSITORY>.git
git push --all winesapos
git push --tags winesapos

Build Packages for winesapOS Repository

A container and script are provided to pre-build important AUR packages for winesapOS.

cd scripts/repo/
docker build --no-cache --tag ekultails/winesapos-build-repo:latest .
mkdir /tmp/winesapos-build-repo
docker run --name winesapos-build-repo --rm --volume /tmp/winesapos-build-repo:/output ekultails/winesapos-build-repo:latest

Those packages are then hosted on a Kubernetes cluster with the following requirements:

Apply all of the Kubernetes manifests to create NGINX containers on the Kubernetes cluster.

kubectl apply -f scripts/repo/k8s/

For copying new packages over, temporarily set deployment.spec.template.spec.containers.volumeMounts[0].readOnly to false.

kubectl --namespace winesapos-repo edit deployment deploy-winesapos-repo

Then copy the new files into one of the containers. A single persistent volume claim is shared among all of the containers.

kubectl --namespace winesapos-repo cp <PACKAGE_FILE> deploy-winesapos-repo-<UUID>:/usr/share/nginx/html/

Release

  1. Add upgrade notes to the UPGRADES.md file.

  2. For a new release, update the VERSION file in the git repository with the new version before building an image.

  3. After a build, make sure that no tests are failing by looking for lines that end with "FAIL".

    $ grep -P 'FAIL$' /winesapos/etc/winesapos/winesapos-install.log

  4. On the hypervisor, stop the virtual machine and then sanitize the image.

    $ sudo virt-sysprep --operations defaults,-customize -a /var/lib/libvirt/images/winesapos.img

  5. Create a release by using the universal zip compression utility. Using zip also allows for splitting the archive into 2 GiB parts which is required for uploading a GitHub release. Do this for both a build of the "performance" (default) and "secure" images.

    $ cd /var/lib/libvirt/images/
$ sudo mv winesapos.img winesapos-[performance|secure]-<VERSION>.img
$ sudo zip -s 1900m winesapos-[performance|secure]-<VERSION>.img.zip winesapos-[performance|secure]-<VERSION>.img
$ ls -1 | grep winesapos
winesapos-[performance|secure]-<VERSION>.img
winesapos-[performance|secure]-<VERSION>.img.z01
winesapos-[performance|secure]-<VERSION>.img.z02
winesapos-[performance|secure]-<VERSION>.img.zip

  6. Create SHA512 checkums separately for both the performance and secure image and their related archive files. Users can then use those files to check for corruption or tampering.

    $ sha512sum winesapos-performance* > winesapos-performance-<VERSION>_sha512sum.txt
$ sha512sum winesapos-secure* > winesapos-secure-<VERSION>_sha512sum.txt
$ sha512sum --check winesapos-performance-<VERSION>_sha512sum.txt
$ sha512sum --check winesapos-secure-<VERSION>_sha512sum.txt

  7. Create a git tag and push it.

    $ git tag X.Y.Z
$ git push origin X.Y.Z

  8. Sync the stable branch with the latest tag. This is required for the upgrade script. Old versions of winesapOS will pull the latest upgrade script from the stable branch.

    $ git checkout stable
$ git rebase X.Y.Z
$ git push origin stable