Visual Regression Testing is a software testing technique that focuses on detecting visual changes and differences in the user interface of web applications or websites. It involves capturing screenshots of web pages at different stages of development and comparing them to identify any unexpected visual regressions caused by code changes or updates.
Baseline Screenshots
The main goal of Ant Design's visual regression testing is to detect visual changes in components and avoid visual issues introduced by PR changes. We use jest-puppeteer as our testing framework. By combining Puppeteer with Jest, we take screenshots of each component demo and compare them with baseline screenshots.
You can find visual regression test code in __tests__/image.test.ts under each component. You can run visual screenshots in the antd repository using the following command:
npm run test:image # Screenshots will be saved in the imageSnapshots directory. For specific component screenshots, use: npm run test:image -- components/button
Visual Regression Solutions
Argos
Initially, we used Argos as our visual regression testing solution. However, Argos changed their pricing strategy, and with antd triggering visual regression tests on every PR, comparing nearly 6,000 screenshots each time, the cost became unsustainable for us.
Self-hosted
We built our own visual regression testing solution using jest-puppeteer mentioned earlier. We take screenshots of each component demo using four themes: dark, light, compact, and cssVar, then upload these screenshots to Alibaba Cloud OSS as baseline screenshots.
Using GitHub Actions for continuous integration, we automatically capture and upload screenshots to OSS whenever the base branch code changes, ensuring the baseline screenshots stay up-to-date.
For branches requiring visual regression testing, we use pixelmatch to compare current screenshots with baseline screenshots. If differences are found, difference screenshots are generated, and the difference report is uploaded to OSS.
Further leveraging GitHub Actions, we implement baseline screenshot comparison in PRs. If visual differences are detected, the CI uploads the difference screenshots and report to OSS, displays the visual differences in the PR, and marks it as failed, requiring developers to fix the issues.
Local Visual Regression Testing
When developing locally and preparing to submit a PR contribution, we can run visual regression tests in advance using the following command:
npm run test:visual-regression:local # Follow the prompts to select components for visual regression testing
