Why your PR shows a change you did not make

A visual regression baseline is the accepted screenshot a new capture is diffed against. Everything about visual testing hinges on that one reference image being the right one. Get it right and a diff shows only what your change did. Get it wrong and the tool becomes noise, and the noise is specific: changes in your PR that you never made.

If you have used one of these tools, you have probably said a version of this: it compares against main, and once main has moved ahead it shows you a diff that does not exist, because it is not comparing from the point where your branch was based off. Purely noise. It is the most common reason people give for disliking the whole category, and they are right to.

The same thing from the other side: dozens of open pull requests, two people editing the same page, one merges, and now the other has to rebase and re-run the check because the reference moved under them. Both of them are describing a correctness bug in baseline resolution, and it is worth being precise about why, because the fix is not obvious.

A flake is non-determinism in the render: the same commit produces different pixels. A phantom diff is a real pixel difference between two correct renders of two different commits, surfaced in the wrong PR. Freezing the clock will not fix a baseline pointed at the wrong commit. They need different fixes, and a tool that reports both as "changed" trains you to ignore both.

The right reference is the last accepted image on a commit your branch actually contains. Not the tip of main, and not a merge-base guess either. UI Verify walks your branch's own git ancestry backwards: the CLI confirms which candidate commits are really ancestors of your head, and the server takes the most recently accepted image among them. You can see this happen in the upload log; on our own main branch this week it printed Confirmed 268 / 506 baseline commits in git ancestry, which is the CLI intersecting the server's candidate list against local git.

Accepted images on main are admitted per story only once your branch contains the commit that accepted them. That single rule is what stops a branch that is merely behind main from diffing against main's drifted reference. When a PR merges, its accepted baselines are copied onto the default branch at the merge commit, and every branch that later pulls that commit inherits them instead of re-flagging the same intended change. The full walk, including what happens on squash and rebase, is in how baselines are resolved.

One consequence I only appreciated after building it: a pull-request build in GitHub Actions runs at a synthetic merge commit that exists in no one's history. So the PR's accepts would be invisible to an ancestry walk under every merge strategy, and the merge-time copy above is how a PR's accept reaches main at all. If a tool you are evaluating cannot explain what it does at that step, that is where the phantom diffs are coming from.

When you accept a changed capture, it becomes the baseline for that story on that branch, and the next build diffs against what you approved. Accepting is how you move the definition of correct forward on purpose. We also made the very first build of a project block for review rather than auto-accept, after an early version of our own product silently baked a loading spinner into a baseline nobody had looked at. The first N baselines are the ones everyone inherits, so they are the last ones that should be set without a human seeing them.

In the repository that builds UI Verify there is a written rule for reviewing our own visual builds: a changed story you cannot attribute to your own diff is an alert, not a triage decision. Do not accept it, do not call it flake, do not re-run and see. The likely explanation is that a baseline was carried forward that had in fact moved, and that failure is silent by construction: the build goes green and the wrong image propagates to every branch that inherits it. A false alarm costs one investigation. A missed one costs every baseline downstream.

That is the practical reason baseline resolution is not a detail. A reviewer who opens a PR and sees a wall of other people's merges learns that the check is noise, skims past it, and the one build with a real regression buried in that wall looks exactly like the false ones. I run our check on every pull request and again on main after merge, so the rare case of two branches editing the same file still gets a final picture. Correct baselines also make skip-unchanged safe: a story can be carried forward without re-rendering only because the tool knows its reference has not moved. When resolution is wrong, skipping is dangerous; when it is right, skipping is just efficiency.