Visual testing on open source repos and fork pull requests
Run visual regression testing on an open source repo where PRs come from forks: the maintainer approval flow GitHub requires, and the CI setup that avoids common hangs.
Open source repos get most of their pull requests from forks, and GitHub deliberately withholds secrets from a fork's workflow run. A visual check uploads with a project API key, so on a fork PR it has nothing to authenticate with and cannot run on its own. This is the flow GitHub does support, plus the two CI gotchas that cost the most time to debug.
Why a fork pull request cannot run the visual check automatically
GitHub never passes repo secrets to a workflow triggered by a pull_request event from a fork - only a read-only token - so a stranger's pull request cannot read your API key. The capture runs in the contributor's CI, which has no key, so the upload step has nothing to authenticate with and the check does not run. This is a GitHub security rule, not a UI Verify limit: every tool that uploads with a secret hits the same wall.
Approve the visual check with a maintainer gate
The fix is a maintainer-approval gate: a trusted person reviews the diff, then authorizes a run that has the key. GitHub gives you the building blocks.
- In the repo settings, under Actions, require approval for workflows from outside collaborators so a maintainer clicks Approve and run before anything executes on a fork PR.
- Run the upload from a
pull_request_targetworkflow, which executes in the base-repo context where secrets are available, and check out the pull request head to capture the contributor's code. - Gate that job behind a maintainer action - a
safe-to-testlabel, or a GitHub Environment with required reviewers - so only reviewed code ever reaches the step that holds the key.
pull_request_target runs with your secrets, so treat it like production: only ever run code a maintainer has reviewed. An unapproved fork PR must never reach the upload step, or a malicious change could read the key.Run CI in the Playwright container so it does not hang on apt
The capture job installs a browser, and npx playwright install --with-deps shells out to apt-get for Chromium's system libraries. On GitHub's Ubuntu runners that apt call regularly hangs, on a slow mirror or an apt lock, with no timeout - the job runs until the global limit. Run the job inside Microsoft's Playwright image instead: Chromium and its system libraries are pre-baked, so there is no apt at all.
jobs:
visual:
runs-on: ubuntu-latest
# Chromium + its system libs are pre-baked, so no apt-get to hang on.
# Match the tag to your @playwright/test version.
container:
image: mcr.microsoft.com/playwright:v1.55.0-nobleWhy does git say dubious ownership in the container?
Inside a container the checked-out repo is owned by a different user than the job runs as, so git refuses to run in it and prints fatal: detected dubious ownership. The upload reads the commit and branch for the build, so it fails. Add the config git itself suggests, right after checkout:
- run: git config --global --add safe.directory "$GITHUB_WORKSPACE"Keep captures deterministic without a staging backend
A fork contributor cannot reach a private staging backend - no VPN, no secrets - so a test that renders live data will flake for them and for you as the data drifts. Freeze the data in the test instead: intercept the API with committed fixtures, or drive the components in Storybook, so every capture replays identical content regardless of where it runs. See Fix flaky visual tests.
Deterministic Playwright captures
Stop real-page diffs that flake without a real change.
Visual testing for agents
UI Verify captures your UI on every pull request and an AI judge tells an intended change from a real regression. See how it works.
Get started