Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically updated nixos channel pins #252057

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
# GitHub actions
/.github/workflows @NixOS/Security @Mic92 @zowoq
/.github/workflows/merge-staging @FRidh
/.github/workflows/channel-pin.yml @infinisil

# EditorConfig
/.editorconfig @Mic92 @zowoq
Expand All @@ -29,6 +30,7 @@
/lib/debug.nix @edolstra @Profpatsch
/lib/asserts.nix @edolstra @Profpatsch
/lib/path.* @infinisil @fricklerhandwerk
/lib/channel.* @infinisil

# Nixpkgs Internals
/default.nix @Ericson2314
Expand Down
118 changes: 118 additions & 0 deletions .github/workflows/channel-pin.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
name: Update channel pins

on:
push:
branches:
- nixos-unstable
# Any release branches like nixos-23.05
- 'nixos-[0-9][0-9].[0-9][0-9]'
Comment on lines +7 to +8
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Any release branches like nixos-23.05
- 'nixos-[0-9][0-9].[0-9][0-9]'
# Any release branches like nixos-23.05
- 'nixos-[0-9][0-9].[0-9][0-9]'
# There is code below that assumes any branch not named `nixos-unstable`
# matches the pattern above; be sure to update that code if you add to this list.


# cancel any other workflows in progress
concurrency:
group: ${{ github.ref }}
cancel-in-progress: true

# Needed to create PRs
permissions:
contents: write
pull-requests: write

jobs:
update_pin:
name: Update channel pin
runs-on: ubuntu-latest
steps:
- uses: cachix/install-nix-action@v22
- name: Compute development branch
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- name: Compute development branch
- name: Compute name of base branch for newly-created PR

id: dev-branch
run: |
if [[ "$GITHUB_REF_NAME" == nixos-unstable ]]; then
branch=master
else
# Removes the "nixos" prefix and replaces it with "release"
branch=release${GITHUB_REF_NAME#nixos}
fi
echo "branch=$branch" >> "$GITHUB_OUTPUT"
- name: Check out development branch
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- name: Check out development branch
- name: Check out base branch for newly-created PR

uses: actions/checkout@v3
with:
ref: ${{ steps.dev-branch.outputs.branch }}
- name: Update pin
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- name: Update pin
- name: Create commit and prepare PR which will update the pin

id: update
run: |
newRev=$GITHUB_SHA
pinFile=lib/channel/pin.json

echo "Fetching new revision $newRev"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
echo "Fetching new revision $newRev"
echo "Fetching tarball for new git revision $newRev of nixpkgs"

stdout=$(nix-prefetch-url \
"${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/tarball/$newRev" \
--type sha256 --unpack --print-path --name nixpkgs)
mapfile -t newInfo <<<"$stdout"
newHash=${newInfo[0]}
newPath=${newInfo[1]}
newPinFileContents=$(jq -n \
--arg rev "$newRev" \
--arg sha256 "$newHash" \
'$ARGS.named')

echo -e "File $pinFile would be updated to:\n$newPinFileContents"

echo "Comparing this with the revision of the existing file"
if ! oldRev=$(jq -r '.rev' "$pinFile"); then
echo "There is no existing file, make sure to initialize it properly, possibly using the above value"
exit 1
else
echo "The existing file has revision $oldRev, now fetching that too"
stdout=$(nix-prefetch-url \
"${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/tarball/$oldRev" \
--type sha256 --unpack --print-path --name nixpkgs)
mapfile -t newInfo <<<"$stdout"
oldHash=${oldInfo[0]}
oldPath=${oldInfo[1]}

change_url="$GITHUB_SERVER_URL"/"$GITHUB_REPOSITORY"/compare/"$oldRev".."$newRev"

echo "Checking if anything other than $pinFile changed between $oldRev and $newRev"
# Only don't make a PR if only the pin file changed, not if it was added/removed
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Only don't make a PR if only the pin file changed, not if it was added/removed
# Don't make a PR if the pinfile already existed and has changed but nothing else has changed.

if [[ -f "$oldPath"/"$pinFile" ]] \
&& [[ -f "$newPath"/"$pinFile" ]] \
&& diff --recursive --exclude "$pinFile" "$oldPath" "$newPath"; then
echo "Nothing changed, no PR to update the pin necessary"
create_pr=
else
echo "The channel changed, PR to update the pin is necessary"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
echo "The channel changed, PR to update the pin is necessary"
echo "The channel changed, and there have been non-channel-pin changes, so a PR to update the pin is necessary"

create_pr=1
fi
fi
echo "create_pr=$create_pr" >> "$GITHUB_OUTPUT"

if [[ -n "$create_pr" ]]; then
echo "Updating $pinFile"
printf "%s\n" "$newPinFileContents" > "$pinFile"

echo "Assembling PR title and body"
if [[ "$GITHUB_REF_NAME" != nixos-unstable ]]; then
pr_title="[${GITHUB_REF_NAME#nixos-}] "
fi
pr_title="${pr_title}Update pinned channel commit"

pr_body_path=$(mktemp)
{
echo "Automated PR to update the pin of the $GITHUB_REF_NAME channel in the ${{ steps.dev_branch.outputs.branch }} branch to the latest commit $GITHUB_SHA."
echo ""
echo "[Channel changes]($change_url)"
} > "$pr_body_path"

echo "pr_title=$pr_title" >> "$GITHUB_OUTPUT"
echo "pr_body_path=$pr_body_path" >> "$GITHUB_OUTPUT"
fi
- name: Create Pull Request
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- name: Create Pull Request
- name: Open the Pull Request prepared in the previous step

uses: peter-evans/create-pull-request@v5
if: ${{ steps.update.outputs.create_pr != '' }}
with:
branch: "update-channel-pin/${{ steps.dev-branch.outputs.branch }}"
commit-message: "Update pinned channel commit"
title: "${{ steps.update.outputs.pr_title }}"
author: "GitHub <[email protected]>"
body-path: "${{ steps.update.outputs.pr_body_path }}"

4 changes: 4 additions & 0 deletions lib/channel/default.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{ lib }:
{
latestKnownNixOSChannelInfo = lib.importJSON ./pin.json;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
latestKnownNixOSChannelInfo = lib.importJSON ./pin.json;
# This pin is updated to point to the most recent completely-successful
# build of the Hydra channel which has the maximal set of pnames (i.e.
# advances most conservatively). Currently that Hydra channel is named
# "nixos-unstable".
latestKnownHydraChannelInfo = lib.importJSON ./pin.json;

I still remember how tremendously confusing it was to me that the nixpkgs-unstable channel actually tracks darwin, whereas the nixos channel is the one you want to follow if you use linux even if you don't use nixos.

Obviously changing the channel names is not on the table at this point, but maybe we can try to limit the propagation of this terribly-confusing naming scheme?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the nixpkgs-unstable channel actually tracks darwin

Huh I don't think so? I don't see anything Darwin-specific for that channel anywhere in the Hydra setup.

}
4 changes: 4 additions & 0 deletions lib/channel/pin.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"rev": "a999c1cc0c9eb2095729d5aa03e0d8f7ed256780",
"sha256": "178smvv8f8pashdjcr9bhmp0baji0lhfcxqy3cn7m19g8rgd6539"
}
2 changes: 2 additions & 0 deletions lib/default.nix
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,8 @@ let
# linux kernel configuration
kernel = callLibs ./kernel.nix;

channel = callLibs ./channel;

inherit (builtins) add addErrorContext attrNames concatLists
deepSeq elem elemAt filter genericClosure genList getAttr
hasAttr head isAttrs isBool isInt isList isPath isString length
Expand Down