UI Verify
All docs
Recipes10 min readUpdated

Visual testing on open source repos and fork pull requests

Run visual checks on pull requests from forks without exposing your API key: the two-workflow GitHub Actions setup, and why not pull_request_target.

Open source repos get many of their pull requests from forks, and GitHub withholds your secrets from a fork's workflow run. The usual one-job setup (capture, then uiverify upload with UIVERIFY_API_KEY) therefore captures fine on a fork PR but has no key to upload with. The fix is two workflows: the first captures with no secrets, the second uploads that capture with your key and never runs anything from the pull request. Pushes and pull requests from your own branches keep working exactly as before.

Why a fork pull request cannot upload on its own

GitHub never passes repository secrets to a workflow triggered by a pull_request event from a fork, even after a maintainer approves the run. It gets only a read-only token, so a stranger's pull request cannot read your API key. This is a GitHub security rule, not a UI Verify limit: every tool that uploads with a secret hits the same wall. Adding the secret to the fork does not help either, because the run executes in your repo and reads only your repo's secrets.

How the two-workflow setup works

  1. Your visual workflow runs on pull_request and push as usual. For a push or a pull request from your own branches it uploads in the same job, with your key. For a pull request from a fork it saves the capture as a build artifact instead.
  2. A second workflow, triggered by workflow_run when the first one succeeds, downloads that artifact and uploads it for the pull request's own commit, branch and number. GitHub always runs this workflow from your default branch's copy, with your secrets, so a pull request cannot change what it does.
  3. UI Verify posts its UI Verify: <slug> status on the pull request's commit, and accepting the changes turns it green, the same as for any other pull request.

1. The visual workflow

Start from your quickstart workflow and add two conditions: upload only when the run is not from a fork, and save the capture as an artifact when it is. Replace the capture step with yours: build Storybook into storybook-static, or run your Vitest or Playwright tests, which write uiverify-archive.

.github/workflows/visual.yml
name: Visual tests
on:
  push:
    branches: [main] # your default branch
  pull_request:

jobs:
  visual:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
        with:
          fetch-depth: 0
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - run: npm run test:visual # your capture, writes ./uiverify-archive

      # Your own branches and pushes: upload here, with the key.
      - name: Upload to UI Verify
        if: github.event_name == 'push' || github.event.pull_request.head.repo.full_name == github.repository
        run: |
          FLAGS="--static-dir ./uiverify-archive --only-changed"
          if [ "$GITHUB_EVENT_NAME" = "push" ]; then FLAGS="$FLAGS --auto-accept-changes"; fi
          npx -y uiverify@1.7.0 upload $FLAGS
        env:
          UIVERIFY_API_KEY: ${{ secrets.UIVERIFY_API_KEY }}

      # A fork's pull request has no key: save the capture for visual-upload.yml.
      - name: Save the capture
        if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name != github.repository
        uses: actions/upload-artifact@v4
        with:
          name: uiverify-archive
          path: ./uiverify-archive/
          if-no-files-found: error
          retention-days: 7
--auto-accept-changes on the default-branch push matters more on an open source repo. Accepting changes on a fork's pull request approves that pull request; once it merges, the push run on your default branch accepts the merged result as the new baseline, so the next pull request diffs against it.

Dependabot pull requests need the key as a Dependabot secret

Dependabot opens its pull requests from a branch in your own repo, so the visual workflow treats them as yours and uploads in the same job. GitHub gives a workflow triggered by Dependabot only the secrets stored for Dependabot, not your Actions secrets, so UIVERIFY_API_KEY is empty and the upload fails with UIVERIFY_API_KEY is required. Add the same key under Settings, Secrets and variables, Dependabot. The check is worth keeping on these pull requests: a dependency bump is one of the most common ways a UI changes without anyone touching your code.

2. The upload workflow for fork pull requests

Add this file as is. The workflows: name must match the name: of the visual workflow above. It downloads the capture, fetches the pull request's commits as git objects (the CLI reads them to find the right baseline) without checking out a single file, and runs a pinned CLI from an empty directory.

.github/workflows/visual-upload.yml
name: Visual upload

# Runs from the default branch with secrets, after "Visual tests" finishes on a fork's
# pull request. Never runs code from the pull request: no checkout of its files, no
# npm ci, no npx inside its tree. Branch names reach the shell only through env.
on:
  workflow_run:
    workflows: [Visual tests]
    types: [completed]

permissions:
  contents: read
  actions: read
  pull-requests: read

jobs:
  upload:
    if: >
      github.event.workflow_run.conclusion == 'success' &&
      github.event.workflow_run.event == 'pull_request' &&
      github.event.workflow_run.head_repository.full_name != github.repository
    runs-on: ubuntu-latest
    steps:
      - uses: actions/setup-node@v4
        with:
          node-version: 24

      - uses: actions/download-artifact@v4
        with:
          name: uiverify-archive
          path: ${{ runner.temp }}/archive
          run-id: ${{ github.event.workflow_run.id }}
          github-token: ${{ github.token }}

      - name: Find the pull request
        id: meta
        env:
          GH_TOKEN: ${{ github.token }}
          REPO: ${{ github.repository }}
          HEAD_BRANCH: ${{ github.event.workflow_run.head_branch }}
          HEAD_OWNER: ${{ github.event.workflow_run.head_repository.owner.login }}
        run: |
          set -euo pipefail
          # workflow_run lists no pull requests for a fork, so look it up by its head.
          # -f encodes the branch name, which may contain characters like "&".
          pr=$(gh api -X GET "repos/$REPO/pulls" -f state=open -f "head=$HEAD_OWNER:$HEAD_BRANCH" --jq '.[0].number // empty')
          {
            echo "pr=$pr"
            echo "branch<<__EOF__"
            echo "$HEAD_OWNER:$HEAD_BRANCH" # a fork's "main" must not look like yours
            echo "__EOF__"
          } >> "$GITHUB_OUTPUT"

      - name: Fetch the commit's history (objects only, no checkout)
        if: steps.meta.outputs.pr != ''
        env:
          REPO: ${{ github.repository }}
          HEAD_SHA: ${{ github.event.workflow_run.head_sha }}
        run: |
          set -euo pipefail
          git init -q "$RUNNER_TEMP/repo"
          cd "$RUNNER_TEMP/repo"
          git remote add origin "https://github.com/$REPO"
          git fetch -q --filter=blob:none --no-tags origin "$HEAD_SHA"
          git update-ref HEAD "$HEAD_SHA"

      - name: Upload to UI Verify
        if: steps.meta.outputs.pr != ''
        env:
          UIVERIFY_API_KEY: ${{ secrets.UIVERIFY_API_KEY }}
          COMMIT_SHA: ${{ github.event.workflow_run.head_sha }}
          BRANCH: ${{ steps.meta.outputs.branch }}
          PR_NUMBER: ${{ steps.meta.outputs.pr }}
        run: |
          set -euo pipefail
          mkdir -p "$RUNNER_TEMP/clean" && cd "$RUNNER_TEMP/clean"
          npx -y uiverify@1.7.0 upload \
            --static-dir "$RUNNER_TEMP/archive" \
            --working-directory "$RUNNER_TEMP/repo" \
            --only-changed --exit-zero-on-changes
A workflow_run workflow only runs from the copy on your default branch, so it starts working for fork pull requests once this file is merged. The pull request that adds it cannot exercise it.

Several visual suites share one upload workflow

A repo with more than one visual suite, such as component tests and docs pages that each upload to their own UI Verify project, needs only one upload workflow. List every capture workflow under workflows: and pick the key by the name of the run that finished. Each capture workflow keeps saving its artifact as uiverify-archive; the download step reads it from the run that triggered the upload (run-id), so the suites never mix.

.github/workflows/visual-upload.yml
on:
  workflow_run:
    workflows: [Visual tests, Docs visual tests]
    types: [completed]

# ...the same job as above, with the key picked per suite:
      - name: Upload to UI Verify
        if: steps.meta.outputs.pr != ''
        env:
          UIVERIFY_API_KEY: ${{ github.event.workflow_run.name == 'Docs visual tests' && secrets.UIVERIFY_DOCS_API_KEY || secrets.UIVERIFY_API_KEY }}

3. Gate the merge on the UI Verify status

A workflow_run run is recorded against your default branch, not the pull request, so it never shows as a job on the PR. That is why it passes --exit-zero-on-changes. On a fork's pull request the gate is the UI Verify: <slug> status, which flips green the moment the changes are accepted. Make it a required status check; for your own branches, keep the job gate and the auto-rerun workflow if you prefer it. Both are in Make the visual check required.

Keep GitHub's approval gate on too: in the repo settings, under Actions, require approval for workflows from outside collaborators, so a maintainer clicks Approve and run before a first-time contributor's capture runs at all.

What keeps the key safe

  • The job that runs the contributor's code (the capture) has no secrets.
  • The job that holds the key runs none of the contributor's code: it never checks out the pull request, never runs npm ci, and never runs npx inside the pull request's tree, where an edited .npmrc or package.json could swap the CLI for something else.
  • The CLI version is pinned and runs from an empty directory, and it only reads the downloaded capture plus git objects.
  • Values the pull request controls, like its branch name, reach the shell through env, never through ${{ }} inside a script, where a crafted branch name could inject a command.
  • A pull request that edits either workflow changes nothing that runs with the key: GitHub runs workflow_run from your default branch's copy.
The capture itself comes from the contributor's code, so a pull request can make its own screenshots show anything. That is true of any CI check, and it is why a fork's result is input for your review, and baselines on your default branch only move from the run after you merge.

Why not pull_request_target?

pull_request_target also runs your default branch's workflow file with secrets for a fork's pull request, and it is the setup many guides suggest. To capture the pull request's UI, though, it has to check out and run the contributor's code: install scripts, .npmrc, the test config and the tests themselves. Doing that in a job that holds your key lets a malicious pull request read it, which GitHub Security Lab calls a pwn request. Splitting it into two jobs helps, but the capture job still runs a stranger's code with your repo's privileges: outside-contributor approval does not apply to pull_request_target, the token can write unless you restrict it, and the job shares the Actions cache with your default branch. With pull_request plus workflow_run, GitHub sandboxes the untrusted half for you.

What if installing dependencies needs a secret?

If your build pulls packages from a private registry, a fork's pull request cannot install them, so no visual check can capture it in GitHub Actions, with or without this setup. Run the visual tests in the CI that already holds the registry token and upload from there; the CLI reads the commit, branch and pull request from COMMIT_SHA, BRANCH and PR_NUMBER on any CI. Whether that pipeline should build outside contributions is your call, since it means running their code next to the token.

Run CI in the Playwright container so it does not hang on apt

The capture job installs a browser, and npx playwright install --with-deps shells out to apt-get for Chromium's system libraries. On GitHub's Ubuntu runners that apt call regularly hangs, on a slow mirror or an apt lock, with no timeout - the job runs until the global limit. Install only the browser you capture with (chromium), or run the job inside Microsoft's Playwright image, where Chromium and its system libraries are pre-baked, so there is no apt at all.

.github/workflows/visual.yml
jobs:
  visual:
    runs-on: ubuntu-latest
    # Chromium + its system libs are pre-baked, so no apt-get to hang on.
    # Match the tag to your @playwright/test version.
    container:
      image: mcr.microsoft.com/playwright:v1.55.0-noble

Why does git say dubious ownership in the container?

Inside a container the checked-out repo is owned by a different user than the job runs as, so git refuses to run in it and prints fatal: detected dubious ownership. The upload reads the commit and branch for the build, so it fails. Add the config git itself suggests, right after checkout:

yaml
- run: git config --global --add safe.directory "$GITHUB_WORKSPACE"

Keep captures deterministic without a staging backend

A fork contributor cannot reach a private staging backend - no VPN, no secrets - so a test that renders live data will flake for them and for you as the data drifts. Freeze the data in the test instead: intercept the API with committed fixtures, or drive the components in Storybook, so every capture replays identical content regardless of where it runs. See Fix flaky visual tests.

Visual testing for agents

UI Verify captures your UI on every pull request and an AI judge tells an intended change from a real regression. See how it works.

Get started