Skip to content

Commit

Permalink
fix: move input setup to inline bash (#145)
Browse files Browse the repository at this point in the history
Because of path shenanigans when running the trunk-action in a
container, the input-setup action wasn't getting found by node. To avoid
this, I've rewritten the logic in bash. This PR also fixes TRUNK-7633
and adds tests for running the action in a container, so we don't get
blindsided by something like this again.

Workflow dispatch code path was tested manually with the services cli
tool
[here](https://github.com/trunk-io/.trunk-internal/actions/runs/5350157016/jobs/9702379590).

-----
Some context on the things I tried prior to this:
- I tried using `$GITHUB_ACTION_PATH` instead of `${{
github.action_path}}` to find the action, as the environment variable is
set correctly, but the context variable is not. However, using the env
var, github fails to find the action at all. Using the context variable,
github finds the `action.yml` and `mask.js` files, but passes an
incorrect path into node to run.
- I considered calling the subaction directly with `uses:
trunk-io/trunk-action/input-setup@v1` - however, this would potentially
introduce a version mismatch between the version of the action being run
and the version of the subaction being wrong
- I considered simply calling `node mask.js` inline in the composite
action, but didn't want to add a setup node action that would
potentially conflict with another node install
- I tried to use templating in the main filename javascript action to
pass in the correct path - while you can use templating in that field,
you cannot access the `env` or `input` contexts, making it impossible to
set the field correctly.
  • Loading branch information
puzzler7 committed Jun 22, 2023
1 parent 35fcebf commit f15b146
Show file tree
Hide file tree
Showing 7 changed files with 218 additions and 210 deletions.
117 changes: 117 additions & 0 deletions .github/workflows/docker_repo_tests.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
name: docker_repo_tests
on: [workflow_dispatch, workflow_call]

jobs:
repo_tests:
name: ${{ matrix.repo }} ${{ matrix.description }}
runs-on: ubuntu-latest
# This is a docker image that mimics the github runner image
# https://github.com/catthehacker/docker_images/pkgs/container/ubuntu
container: catthehacker/ubuntu:runner-20.04
strategy:
fail-fast: false
matrix:
include:
# Items in this list satisfy a few criteria:
#
# * test has to be useful/interesting and add value atop the monorepo tests for
# trunk init
#
# * the repo has to exercise some functionality specific to action.yaml (e.g. our
# custom Node functionality for npm/yarn/pnpm)
#
# * the repo and its dependency closure should be fast to set up, since we trigger
# this workflow on PRs
#
- repo: highlightjs/highlight.js
ref: 4f9cd3bffb6bc55c9e2c4252c7b733a219880151
description: (uses npm)
post-init: |
# terrascan scans dockerfile.js (a JS file for parsing dockerfiles) as a dockerfile itself
echo "src/languages/dockerfile.js" >> .gitignore
cp local-action/repo_tests/highlightjs.yaml .trunk/user.yaml
- repo: pallets/flask
ref: 4bcd4be6b7d69521115ef695a379361732bcaea6
post-init: |
# bandit is outputting a progress bar (TODO: TRUNK-7656)
${TRUNK_PATH} check disable bandit
# prettier chokes on this malformed html file
echo "examples/celery/src/task_app/templates/index.html" >> .gitignore
cp local-action/repo_tests/flask.yaml .trunk/user.yaml
- repo: postcss/postcss
ref: aa9e03ea4708909631eba70500c8c0cc0708bb4e
description: (uses pnpm)
post-init: |
${TRUNK_PATH} check enable eslint
- repo: postcss/postcss
ref: aa9e03ea4708909631eba70500c8c0cc0708bb4e
description: (compat test for cli 1.0.0)
post-init: |
${TRUNK_PATH} check enable eslint
- repo: sass/sass
ref: 225e176115211387e014d97ae0076d94de3152a1
description: (uses npm)

- repo: terraform-linters/tflint
ref: 9c34a740319e2410094ca2754e5eca860f2d13f5
post-init: |
# golangci-lint needs us to init with a newer go runtime
${TRUNK_PATH} check disable golangci-lint
# tfsec and terrascan scan these malformed test files and error
echo "integrationtest/" >> .gitignore
echo "terraform/test-fixtures" >> .gitignore
- repo: trunk-io/plugins
ref: main

- repo: z-shell/wiki
ref: d6d8b5da28c170b3b226705795412497f7059681
description: (has trunk.yaml)
post-init: |
# stylelint is exiting with error code -11
${TRUNK_PATH} check disable stylelint
steps:
- name: Checkout ${{ matrix.repo }}
uses: actions/checkout@v3
with:
repository: ${{ matrix.repo }}
ref: ${{ matrix.ref }}

- name: Checkout ${{ github.repository }}
uses: actions/checkout@v3
with:
path: local-action

- name: Run trunk-action in ${{ matrix.repo }}
id: trunk
uses: ./local-action/
with:
cache: true
check-mode: all
post-init: ${{ matrix.post-init }}
arguments: --output-file=/tmp/landing-state.json
cache-key: repo_tests/${{ matrix.repo }}
setup-deps: true
env:
TRUNK_CLI_VERSION:
${{ matrix.description == '(compat test for cli 1.0.0)' && '1.0.0' || '' }}
continue-on-error: true

- name: Check for task failures
shell: bash
run: |
python3 local-action/repo_tests/check_for_task_failures.py \
'${{ github.env }}' \
'${{ matrix.repo }}' \
'${{ matrix.description }}'
- name: Upload landing state
uses: actions/upload-artifact@v3
with:
name: ${{ env.landing_state_artifact_name }} landing state
path: .trunk/landing-state.json
4 changes: 4 additions & 0 deletions .github/workflows/pr.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,7 @@ jobs:
repo_tests:
name: Repository tests
uses: ./.github/workflows/repo_tests.yaml

docker_repo_tests:
name: Repository tests (docker)
uses: ./.github/workflows/docker_repo_tests.yaml
20 changes: 13 additions & 7 deletions .github/workflows/repo_tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ jobs:
ref: 4f9cd3bffb6bc55c9e2c4252c7b733a219880151
description: (uses npm)
post-init: |
# we currently fail to run ruff >v0.0.266
${TRUNK_PATH} check disable ruff
# terrascan scans dockerfile.js (a JS file for parsing dockerfiles) as a dockerfile itself
echo "src/languages/dockerfile.js" >> .gitignore
cp local-action/repo_tests/highlightjs.yaml .trunk/user.yaml
- repo: jbeder/yaml-cpp
Expand All @@ -34,8 +34,6 @@ jobs:
post-init: |
# black complains about py2 code
${TRUNK_PATH} check disable black
# we currently fail to run ruff >v0.0.266
${TRUNK_PATH} check disable ruff
mkdir build
cd build
cmake .. -DCMAKE_EXPORT_COMPILE_COMMANDS=ON
Expand All @@ -48,8 +46,10 @@ jobs:
- repo: pallets/flask
ref: 4bcd4be6b7d69521115ef695a379361732bcaea6
post-init: |
# prettier fails because of a malformed html file
${TRUNK_PATH} check disable prettier
# bandit is outputting a progress bar (TODO: TRUNK-7656)
${TRUNK_PATH} check disable bandit
# prettier chokes on this malformed html file
echo "examples/celery/src/task_app/templates/index.html" >> .gitignore
cp local-action/repo_tests/flask.yaml .trunk/user.yaml
- repo: postcss/postcss
Expand All @@ -69,6 +69,9 @@ jobs:
description: (has trunk.yaml)
post-init: |
# replay is on a very old version
# trunk upgrade requires 3 calls to guarantee correctness
${TRUNK_PATH} upgrade
${TRUNK_PATH} upgrade
${TRUNK_PATH} upgrade
trunk-path: node_modules/.bin/trunk

Expand All @@ -95,6 +98,9 @@ jobs:
post-init: |
# golangci-lint needs us to init with a newer go runtime
${TRUNK_PATH} check disable golangci-lint
# tfsec and terrascan scan these malformed test files and error
echo "integrationtest/" >> .gitignore
echo "terraform/test-fixtures" >> .gitignore
- repo: trunk-io/plugins
ref: main
Expand Down Expand Up @@ -135,7 +141,7 @@ jobs:
check-mode: all
trunk-path: ${{ matrix.trunk-path }}
post-init: ${{ matrix.post-init }}
arguments: --output-file=.trunk/landing-state.json
arguments: --output-file=/tmp/landing-state.json
cache-key: repo_tests/${{ matrix.repo }}
setup-deps: true
env:
Expand Down
94 changes: 82 additions & 12 deletions action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -105,21 +105,91 @@ inputs:
runs:
using: composite
steps:
- name: Input setup (1 of 3)
- name: Input setup
shell: bash
run: ln -s ${{ github.action_path }} .trunk-action
run: |
cat >>$GITHUB_ENV <<EOF
GITHUB_TOKEN=${{ github.token }}
EOF
payload() {
if command -v jq >/dev/null; then
jq -r ".inputs.payload | fromjson | .$1 // empty" ${GITHUB_EVENT_PATH}
else
echo "::error::jq not installed on system!"
exit 1
fi
}
# Every inputs.field should be referenced as INPUT_FIELD later in the action. This allows
# the field to be set either as an argument to the github action or via inputs.json.
if [[ "${{ inputs.check-mode }}" = "payload" ]]; then
INPUT_GITHUB_TOKEN=$(payload githubToken)
INPUT_TRUNK_TOKEN=$(payload trunkToken)
echo "::add-mask::${INPUT_GITHUB_TOKEN}"
echo "::add-mask::${INPUT_TRUNK_TOKEN}"
cat >>$GITHUB_ENV <<EOF
INPUT_GITHUB_TOKEN=${INPUT_GITHUB_TOKEN}
INPUT_TRUNK_TOKEN=${INPUT_TRUNK_TOKEN}
TRUNK_TOKEN=${INPUT_TRUNK_TOKEN}
GITHUB_EVENT_PULL_REQUEST_BASE_REPO_OWNER=$(payload pullRequest.base.repo.owner.login)
GITHUB_EVENT_PULL_REQUEST_BASE_REPO_NAME=$(payload pullRequest.base.repo.name)
GITHUB_EVENT_PULL_REQUEST_BASE_SHA=$(payload pullRequest.base.sha)
GITHUB_EVENT_PULL_REQUEST_HEAD_REPO_FORK=$(payload pullRequest.head.repo.fork)
GITHUB_EVENT_PULL_REQUEST_HEAD_SHA=$(payload pullRequest.head.sha)
GITHUB_EVENT_PULL_REQUEST_NUMBER=$(payload pullRequest.number)
INPUT_ARGUMENTS=$(payload arguments)
INPUT_CACHE=$(payload cache)
INPUT_CACHE_KEY=$(payload cacheKey)
INPUT_CACHE_PATH=$(payload cachePath)
INPUT_CHECK_ALL_MODE=$(payload checkAllMode)
INPUT_CHECK_MODE=$(payload checkMode)
INPUT_CHECK_RUN_ID=$(payload checkRunId)
INPUT_DEBUG=$(payload debug)
INPUT_GITHUB_REF_NAME=$(payload targetRefName)
INPUT_LABEL=$(payload label)
INPUT_SETUP_CACHE_KEY=$(payload setupCacheKey)
INPUT_SETUP_DEPS=$(payload setupDeps)
INPUT_TARGET_CHECKOUT=$(payload targetCheckout)
INPUT_TARGET_CHECKOUT_REF=$(payload targetCheckoutRef)
INPUT_TRUNK_PATH=$(payload trunkPath)
INPUT_UPLOAD_LANDING_STATE=$(payload uploadLandingState)
INPUT_UPLOAD_SERIES=$(payload uploadSeries)
EOF
- name: Input setup (2 of 3)
uses: ./.trunk-action/input-setup/
with:
use-payload: ${{ inputs.check-mode == 'payload' }}
env:
MASK_INPUTS: ${{ toJSON(inputs) }}
MASK_GITHUB_EVENT_PR: ${{ toJSON(github.event.pull_request) }}
else
- name: Input setup (3 of 3)
shell: bash
run: rm .trunk-action
cat >>$GITHUB_ENV <<EOF
INPUT_GITHUB_TOKEN=${{ inputs.github-token }}
INPUT_TRUNK_TOKEN=${{ inputs.trunk-token }}
TRUNK_TOKEN=${{ inputs.trunk-token }}
GITHUB_EVENT_PULL_REQUEST_BASE_SHA=${{ github.event.pull_request.base.sha }}
GITHUB_EVENT_PULL_REQUEST_HEAD_REPO_FORK=${{ github.event.pull_request.head.repo.fork }}
GITHUB_EVENT_PULL_REQUEST_HEAD_SHA=${{ github.event.pull_request.head.sha }}
GITHUB_EVENT_PULL_REQUEST_NUMBER=${{ github.event.pull_request.number }}
GITHUB_REF_NAME=${{ github.ref_name }}
INPUT_ARGUMENTS=${{ inputs.arguments }}
INPUT_CACHE=${{ inputs.cache }}
INPUT_CACHE_KEY=trunk-${{ inputs.cache-key }}-${{ runner.os }}-${{ hashFiles('.trunk/trunk.yaml') }}
INPUT_CACHE_PATH=~/.cache/trunk
INPUT_CHECK_ALL_MODE=${{ inputs.check-all-mode }}
INPUT_CHECK_MODE=${{ inputs.check-mode }}
INPUT_CHECK_RUN_ID=${{ inputs.check-run-id }}
INPUT_DEBUG=${{ inputs.debug }}
INPUT_GITHUB_REF_NAME=${{ github.ref_name }}
INPUT_SETUP_DEPS=${{ inputs.setup-deps }}
INPUT_TARGET_CHECKOUT=
INPUT_TARGET_CHECKOUT_REF=
INPUT_LABEL=${{ inputs.label }}
INPUT_SETUP_CACHE_KEY=${{ inputs.cache-key }}
INPUT_TRUNK_PATH=${{ inputs.trunk-path }}
INPUT_UPLOAD_LANDING_STATE=false
INPUT_UPLOAD_SERIES=${{ inputs.upload-series }}
EOF
fi
- name: Checkout
if: env.INPUT_TARGET_CHECKOUT
Expand Down
13 changes: 0 additions & 13 deletions input-setup/action.yaml

This file was deleted.

Loading

0 comments on commit f15b146

Please sign in to comment.