Fix flaky Playwright screenshot tests

I have shipped my share of flaky Playwright screenshot tests, and for a long time I reached for the same fix everyone reaches for: bump the pixel threshold until the red goes away. It buys you about a week. Then a real regression slips through the loosened threshold, and now you have the worst of both worlds, a test that still flakes and no longer catches anything.

Here is the reframe that actually fixed it for me. A flaky screenshot is not a rendering problem, it is a data problem. The diff is not lying. It is faithfully reporting that the input to the render moved between two runs: the clock advanced, an animation was caught on a different frame, an API returned rows in a new order, or the CI box rasterized a font a hair differently than your laptop did. Pin the input and the diff goes quiet for good, not until the next threshold bump.

toHaveScreenshot() is a pixel comparison. Playwright renders the page now, compares it against a baseline PNG captured earlier, and fails when too many pixels differ. That is exactly what you want when the only thing that changed is your code. It is exactly what you do not want when four other things changed too, none of them your code. A screenshot is a pure function of its inputs, so any input you did not pin is a coin flip the test re-tosses on every run.

Relative timestamps are the single most common flake I see. A "2 minutes ago" label, a new Date() in a header, an Intl format of Date.now(): each one renders a different string the moment the clock ticks, and CI is always a few seconds slower than your machine. Freeze the clock before the page renders anything and the whole class disappears in one line.

Use page.clock.install() instead of setFixedTime when the component reads the clock over its own lifetime (a countdown, a live-updating "ago" label) and you want to control ticks. Set it once in a fixture and every test inherits a frozen clock. This one change alone kills most of the flake in a typical suite.

An infinite CSS or JS animation is never at the same frame twice, so the screenshot catches it wherever it happened to be. Playwright can roll CSS animations to their end and disable transitions for you. For anything it cannot reach from outside the page, a <canvas> render loop or a charting library that animates on mount, kill motion at the source with a stylesheet.

One trap worth calling out: use animation: none, not a near-zero animation-duration or animation-iteration-count: 1. A shrunk-but-present animation still leaves the element promoted to its own compositor layer, whose sub-pixel raster differs run-to-run under render load, so it flakes maybe one time in a hundred, which is the worst frequency to debug. Only none de-promotes it to a resting frame deterministically.

A component that renders whatever a real API returns this second is flaky by definition. Intercept the request and return a fixed fixture, so the render is a function of your fixture and nothing else. The same goes for any randomness the UI surfaces: seed generated ids, pin sort orders, stub Math.random.

This is not a niche precaution. Anyone who has built their own visual-regression harness ends up intercepting every API response and pinning the store behind the component for exactly this reason: the goal is to test the display, and the only way to get a stable display is a stable input. Freeze the data per PR and the diff finally means what you want it to mean.

Even with the clock, animations, and data pinned, a screenshot taken on your Mac will not match one taken on the CI Linux box. Font hinting, sub-pixel antialiasing, and the GPU raster path all differ across operating systems, so you get a few pixels of drift that turn a run red for no real reason. The fix is to render every capture in the same environment: a pinned Docker image, or a hosted fleet, so the fonts, the viewport, and the raster path are identical every time.

A team I talked to ran their entire visual suite in Docker for precisely this reason. Their Mac and a colleague's Linux box disagreed by a handful of pixels and turned nearly every run red, so they isolated the render. It works, but it is a rig you now own and maintain: the image, the browser install, and a per-OS baseline for anyone who runs it locally.

When the flake will not stop, the tempting knob is the tolerance: maxDiffPixels, maxDiffPixelRatio, or threshold. Crank the ratio to 2-3% and the sub-pixel noise stops failing the build. It also stops failing on real regressions. A button shifted 4px, a label that now truncates, a grid column that collapsed on a breakpoint: all of those can fit comfortably under a 2-3% budget.

Every fix above is you hunting down one moving input at a time. There is a way to remove the whole moving-data class at once: record the page into a static archive once, then replay that same archive on every run. Archive-replay bakes the DOM and the network responses that produced it into the archive at record time, so a live API, a feature flag, or an endpoint that returns a new order each call cannot shift the picture on replay. There is nothing live left to fetch.

This is how `@uiverify/playwright` captures: you keep your existing Playwright tests, swap the import, and it records an archive that UI Verify replays and screenshots deterministically on a fixed cloud fleet, so the cross-OS baseline problem goes away with it. You still freeze the clock and animations at record time (the archived CSS re-runs on replay), but the live-data flakes are gone by construction rather than by a stylesheet you maintain. And because a human or a coding agent still has to judge the changes that remain, an AI judge tells an intended change from a real regression, so you review decisions, not diffs.

If you want the levers above applied to your suite for you, the Playwright determinism skill walks your coding agent through them one symptom at a time. The deeper how-to, with the Storybook and Vitest variants, lives in Fix flaky visual tests.