Visual testing on open source repos and fork pull requests
Run visual checks on pull requests from forks without exposing your API key: the two-workflow GitHub Actions setup, and why not pull_request_target.
Open source repos get many of their pull requests from forks, and GitHub withholds your secrets from a fork's workflow run. The usual one-job setup (capture, then uiverify upload with UIVERIFY_API_KEY) therefore captures fine on a fork PR but has no key to upload with. The fix is two workflows: the first captures with no secrets, the second uploads that capture with your key and never runs anything from the pull request. Pushes and pull requests from your own branches keep working exactly as before.
Why a fork pull request cannot upload on its own
GitHub never passes repository secrets to a workflow triggered by a pull_request event from a fork, even after a maintainer approves the run. It gets only a read-only token, so a stranger's pull request cannot read your API key. This is a GitHub security rule, not a UI Verify limit: every tool that uploads with a secret hits the same wall. Adding the secret to the fork does not help either, because the run executes in your repo and reads only your repo's secrets.
How the two-workflow setup works
- Your visual workflow runs on
pull_requestandpushas usual. For a push or a pull request from your own branches it uploads in the same job, with your key. For a pull request from a fork it saves the capture as a build artifact instead. - A second workflow, triggered by
workflow_runwhen the first one succeeds, downloads that artifact and uploads it for the pull request's own commit, branch and number. GitHub always runs this workflow from your default branch's copy, with your secrets, so a pull request cannot change what it does. - UI Verify posts its
UI Verify: <slug>status on the pull request's commit, and accepting the changes turns it green, the same as for any other pull request.
1. The visual workflow
Start from your quickstart workflow and add two conditions: upload only when the run is not from a fork, and save the capture as an artifact when it is. Replace the capture step with yours: build Storybook into storybook-static, or run your Vitest or Playwright tests, which write uiverify-archive.
name: Visual tests
on:
push:
branches: [main] # your default branch
pull_request:
jobs:
visual:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0
- run: npm ci
- run: npx playwright install --with-deps chromium
- run: npm run test:visual # your capture, writes ./uiverify-archive
# Your own branches and pushes: upload here, with the key.
- name: Upload to UI Verify
if: github.event_name == 'push' || github.event.pull_request.head.repo.full_name == github.repository
run: |
FLAGS="--static-dir ./uiverify-archive --only-changed"
if [ "$GITHUB_EVENT_NAME" = "push" ]; then FLAGS="$FLAGS --auto-accept-changes"; fi
npx -y uiverify@1.7.0 upload $FLAGS
env:
UIVERIFY_API_KEY: ${{ secrets.UIVERIFY_API_KEY }}
# A fork's pull request has no key: save the capture for visual-upload.yml.
- name: Save the capture
if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name != github.repository
uses: actions/upload-artifact@v4
with:
name: uiverify-archive
path: ./uiverify-archive/
if-no-files-found: error
retention-days: 7--auto-accept-changes on the default-branch push matters more on an open source repo. Accepting changes on a fork's pull request approves that pull request; once it merges, the push run on your default branch accepts the merged result as the new baseline, so the next pull request diffs against it.Dependabot pull requests need the key as a Dependabot secret
Dependabot opens its pull requests from a branch in your own repo, so the visual workflow treats them as yours and uploads in the same job. GitHub gives a workflow triggered by Dependabot only the secrets stored for Dependabot, not your Actions secrets, so UIVERIFY_API_KEY is empty and the upload fails with UIVERIFY_API_KEY is required. Add the same key under Settings, Secrets and variables, Dependabot. The check is worth keeping on these pull requests: a dependency bump is one of the most common ways a UI changes without anyone touching your code.
2. The upload workflow for fork pull requests
Add this file as is. The workflows: name must match the name: of the visual workflow above. It downloads the capture, fetches the pull request's commits as git objects (the CLI reads them to find the right baseline) without checking out a single file, and runs a pinned CLI from an empty directory.
name: Visual upload
# Runs from the default branch with secrets, after "Visual tests" finishes on a fork's
# pull request. Never runs code from the pull request: no checkout of its files, no
# npm ci, no npx inside its tree. Branch names reach the shell only through env.
on:
workflow_run:
workflows: [Visual tests]
types: [completed]
permissions:
contents: read
actions: read
pull-requests: read
jobs:
upload:
if: >
github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.event == 'pull_request' &&
github.event.workflow_run.head_repository.full_name != github.repository
runs-on: ubuntu-latest
steps:
- uses: actions/setup-node@v4
with:
node-version: 24
- uses: actions/download-artifact@v4
with:
name: uiverify-archive
path: ${{ runner.temp }}/archive
run-id: ${{ github.event.workflow_run.id }}
github-token: ${{ github.token }}
- name: Find the pull request
id: meta
env:
GH_TOKEN: ${{ github.token }}
REPO: ${{ github.repository }}
HEAD_BRANCH: ${{ github.event.workflow_run.head_branch }}
HEAD_OWNER: ${{ github.event.workflow_run.head_repository.owner.login }}
run: |
set -euo pipefail
# workflow_run lists no pull requests for a fork, so look it up by its head.
# -f encodes the branch name, which may contain characters like "&".
pr=$(gh api -X GET "repos/$REPO/pulls" -f state=open -f "head=$HEAD_OWNER:$HEAD_BRANCH" --jq '.[0].number // empty')
{
echo "pr=$pr"
echo "branch<<__EOF__"
echo "$HEAD_OWNER:$HEAD_BRANCH" # a fork's "main" must not look like yours
echo "__EOF__"
} >> "$GITHUB_OUTPUT"
- name: Fetch the commit's history (objects only, no checkout)
if: steps.meta.outputs.pr != ''
env:
REPO: ${{ github.repository }}
HEAD_SHA: ${{ github.event.workflow_run.head_sha }}
run: |
set -euo pipefail
git init -q "$RUNNER_TEMP/repo"
cd "$RUNNER_TEMP/repo"
git remote add origin "https://github.com/$REPO"
git fetch -q --filter=blob:none --no-tags origin "$HEAD_SHA"
git update-ref HEAD "$HEAD_SHA"
- name: Upload to UI Verify
if: steps.meta.outputs.pr != ''
env:
UIVERIFY_API_KEY: ${{ secrets.UIVERIFY_API_KEY }}
COMMIT_SHA: ${{ github.event.workflow_run.head_sha }}
BRANCH: ${{ steps.meta.outputs.branch }}
PR_NUMBER: ${{ steps.meta.outputs.pr }}
run: |
set -euo pipefail
mkdir -p "$RUNNER_TEMP/clean" && cd "$RUNNER_TEMP/clean"
npx -y uiverify@1.7.0 upload \
--static-dir "$RUNNER_TEMP/archive" \
--working-directory "$RUNNER_TEMP/repo" \
--only-changed --exit-zero-on-changesworkflow_run workflow only runs from the copy on your default branch, so it starts working for fork pull requests once this file is merged. The pull request that adds it cannot exercise it.Several visual suites share one upload workflow
A repo with more than one visual suite, such as component tests and docs pages that each upload to their own UI Verify project, needs only one upload workflow. List every capture workflow under workflows: and pick the key by the name of the run that finished. Each capture workflow keeps saving its artifact as uiverify-archive; the download step reads it from the run that triggered the upload (run-id), so the suites never mix.
on:
workflow_run:
workflows: [Visual tests, Docs visual tests]
types: [completed]
# ...the same job as above, with the key picked per suite:
- name: Upload to UI Verify
if: steps.meta.outputs.pr != ''
env:
UIVERIFY_API_KEY: ${{ github.event.workflow_run.name == 'Docs visual tests' && secrets.UIVERIFY_DOCS_API_KEY || secrets.UIVERIFY_API_KEY }}3. Gate the merge on the UI Verify status
A workflow_run run is recorded against your default branch, not the pull request, so it never shows as a job on the PR. That is why it passes --exit-zero-on-changes. On a fork's pull request the gate is the UI Verify: <slug> status, which flips green the moment the changes are accepted. Make it a required status check; for your own branches, keep the job gate and the auto-rerun workflow if you prefer it. Both are in Make the visual check required.
Keep GitHub's approval gate on too: in the repo settings, under Actions, require approval for workflows from outside collaborators, so a maintainer clicks Approve and run before a first-time contributor's capture runs at all.
What keeps the key safe
- The job that runs the contributor's code (the capture) has no secrets.
- The job that holds the key runs none of the contributor's code: it never checks out the pull request, never runs
npm ci, and never runsnpxinside the pull request's tree, where an edited.npmrcorpackage.jsoncould swap the CLI for something else. - The CLI version is pinned and runs from an empty directory, and it only reads the downloaded capture plus git objects.
- Values the pull request controls, like its branch name, reach the shell through
env, never through${{ }}inside a script, where a crafted branch name could inject a command. - A pull request that edits either workflow changes nothing that runs with the key: GitHub runs
workflow_runfrom your default branch's copy.
Why not pull_request_target?
pull_request_target also runs your default branch's workflow file with secrets for a fork's pull request, and it is the setup many guides suggest. To capture the pull request's UI, though, it has to check out and run the contributor's code: install scripts, .npmrc, the test config and the tests themselves. Doing that in a job that holds your key lets a malicious pull request read it, which GitHub Security Lab calls a pwn request. Splitting it into two jobs helps, but the capture job still runs a stranger's code with your repo's privileges: outside-contributor approval does not apply to pull_request_target, the token can write unless you restrict it, and the job shares the Actions cache with your default branch. With pull_request plus workflow_run, GitHub sandboxes the untrusted half for you.
What if installing dependencies needs a secret?
If your build pulls packages from a private registry, a fork's pull request cannot install them, so no visual check can capture it in GitHub Actions, with or without this setup. Run the visual tests in the CI that already holds the registry token and upload from there; the CLI reads the commit, branch and pull request from COMMIT_SHA, BRANCH and PR_NUMBER on any CI. Whether that pipeline should build outside contributions is your call, since it means running their code next to the token.
Run CI in the Playwright container so it does not hang on apt
The capture job installs a browser, and npx playwright install --with-deps shells out to apt-get for Chromium's system libraries. On GitHub's Ubuntu runners that apt call regularly hangs, on a slow mirror or an apt lock, with no timeout - the job runs until the global limit. Install only the browser you capture with (chromium), or run the job inside Microsoft's Playwright image, where Chromium and its system libraries are pre-baked, so there is no apt at all.
jobs:
visual:
runs-on: ubuntu-latest
# Chromium + its system libs are pre-baked, so no apt-get to hang on.
# Match the tag to your @playwright/test version.
container:
image: mcr.microsoft.com/playwright:v1.55.0-nobleWhy does git say dubious ownership in the container?
Inside a container the checked-out repo is owned by a different user than the job runs as, so git refuses to run in it and prints fatal: detected dubious ownership. The upload reads the commit and branch for the build, so it fails. Add the config git itself suggests, right after checkout:
- run: git config --global --add safe.directory "$GITHUB_WORKSPACE"Keep captures deterministic without a staging backend
A fork contributor cannot reach a private staging backend - no VPN, no secrets - so a test that renders live data will flake for them and for you as the data drifts. Freeze the data in the test instead: intercept the API with committed fixtures, or drive the components in Storybook, so every capture replays identical content regardless of where it runs. See Fix flaky visual tests.
Deterministic Playwright captures
Stop real-page diffs that flake without a real change.
Visual testing for agents
UI Verify captures your UI on every pull request and an AI judge tells an intended change from a real regression. See how it works.
Get started