How to Make the Most of Visual Regression Testing
March 21, 2019
Visual Regression is a category of software testing that focuses on identifying visual changes between iterations or versions of a website. It is a process of verifying the reference UI screens with test UI screens and monitoring the changes over time.
Consider a Change Blindness example which is a perceptual phenomenon that occurs when a change in visual stimulus is introduced and the observer does not notice it. For example, observers often fail to notice major differences introduced into an image while it flickers off and on again:
UI Comparator Tool
UI Comparator tool addresses the change blindness phenomenon while comparing screens as part of UI testing. The tool is capable of automatically verifying and detecting the differences. It captures screenshots of Web Pages/UI, compares them with the original/base images and automatically generates a report of differences. Areas of usage include Content Migration (across platforms), Website Migration (across platforms, data centers etc.), CMS Version upgrades, Maintenance Projects (UI Régression), and Development Releases Testing. Non-Viable Areas include Dynamic Web Content Websites.
Here are the best Tips and Tricks on How to Video Record Selenium Test Cases || Our Quality Assurance Team helps our clients deliver tightly integrated insights, technologies, and best practice processes.
Problem Statement and Tool Solution
The “Content Comparison and Verification” phase normally takes a good deal of time and effort as it involves more of manual verification and testing. The following analysis was made from info gathered from recent content migration projects:
50 Web Pages took about 5 hrs. to verify the content manually => Avg. 6 min/page. The same task with tool was done in less than 10 mins. Here is a comparison of manual versus automated UI testing:
|1||A manual “Content Comparison and Verification” phase normally takes a good deal of time and effort as it involves more of manual verification and testing.|
Sample: 50 Web Pages took about 5 Hrs to verify the UI content => Avg. 6 min/Page
|Automated UI comparison is faster by 500% =>|
HIGHER SAVINGS FOR MULTIPLE BUILD / RELEASES
50 Web Pages took about 10 min to verify the UI content => Avg. ~12 sec/Page
|2||Change Blindness is a surprising perceptual phenomenon that occurs when a change in visual stimulus is introduced and the observer does not notice it.|
Manual testers often fail to notice major differences introduced into an image while it flickers off and on again.
|Automated UI Comparison detects minutest of differences|
|3||Content migration projects with dynamic content require Functional testing in addition to UI comparison -> can be cumbersome and time consuming (Takes days to complete)||Automated UI Comparison integrated with functional tests is efficient and can provided segmented reports.|
(Can be run overnight)
- Test and Reference URL’s setup in the CSV file.
- Tool Settings Configured as per the test requirement (Mismatch Threshold, Delay, View Ports for Responsive etc.)
End User Perspective:
Shaded region components would be running in the background and the components outside shaded region would be accessible to end user.
Learn How to Make Continuous Testing the Catalyst in Your Continuous Delivery Process || Sign up today for the latest news about your Digital Transformation.
- Built on robust and validated framework using npm package
- Is Platform agnostic and purely url based
- Automatic visual verification and detection
- Nifty visual report that gives ease of manual re-verification across pages by a non-technical user.
- Supports comparison across multiple resolutions (Responsive)
- Provides option to IGNORE specific aspects/areas of the webpage
- Can run comparison for multiple pages in one go – Batch Run (excel/csv driven)
- Supports scenarios that include ‘actions’
- Configurable (Mismatch threshold, Delay etc.)
Below is a snapshot of test report generated by the tool. Some of the highlights include:
- Provide details of total Pass/Fail
- Side by side comparison of screens for Reference, Test, and Difference of mismatching webpages.
- Provide the details about the percentage of differences for mismatch screens.
UI Comparator tool is largely being used at TA Digital for content migration and CMS upgrade projects. The tool has proven visual testing the websites with thousands of web pages in no time and best of accuracy.
GET HELP FROM OUR EXPERTS
Over the past 19 years, we have completed thousands of digital projects globally. We have one of the largest and deepest multi-solutions digital consulting teams in the world. Our proprietary processes and years of Digital Experience expertise have earned us a 97% customer satisfaction rating with our clients ranging from Global Fortune 1000 to Mid-Market Enterprises, leading educational institutions, and Non-Profits.
DesignRush has recognized TA Digital as a top Web Design Agency.
About TA Digital
TA Digital is an innovative digital transformation agency, specializing in delivering digital experience, commerce, and marketing solutions. For nearly two decades, we have been helping traditional businesses transform and create dynamic digital cultures through disruptive strategies and agile deployment of innovative solutions. We are known as a global leader in the digital technology industry for helping marketing leaders achieve their revenue targets, create profitable, omni-channel customer and commerce experiences. TA Digital has high-level strategic partnerships with digital technology companies Adobe, Microsoft, Sitecore, Acquia, Marketo, SAP Hybris, Elastic Path, IBM Watson Marketing, Coveo and Episerver. The company was named on 2013, 2014, 2015 Inc. 5000 list as one of the fastest-growing technology companies in the United States.