How frequently should I visually test my ecommerce? Short answer: as much as you possibly can. Long answer: it depends. You should consider a minimum of once daily to a maximum frequency of every 15-30 mins.
What factors should I consider to decide the frequency of my visual tests? Let´s start from the scope / objectives of your ecommerce tests. There are 4 main ones to consider for any ecommerce (regardless of size):
- Check that campaign & catalog changes have been applied as expected
- Validate that the latest features have been deployed as expected
- Review commerce platform configurations so that they are properly applied
- Test the ecommerce customer flows and UX are working as expected
With all of the above objectives, you should consider how frequently your online business is updated, with respect to:
- the catalog
- the marketing content
- the back-end code, the user interface (UI), and/or the customer experience (CX)
- the various platform configurations (stock, shipping charges, jurisdictions, taxes, etc.)
Example 1: Test execution of once per day This frequency is appropriate for businesses which update their ecommerce less than once a week. These updates could be seasonal changes related to catalog and stock availability. The business may run a few campaigns per year, but configuration changes such as shipping, taxes, jurisdiction and the code are very rare. Generally the business has a stable ecommerce platform that is never under stress, and dependent on very few (if any) external services. Visual errors may cause high bounce rates and low conversion rates, however the business may deem it acceptable to be aware of such errors within 24 hours.
Example 2: Test execution every 6 hours This frequency is generally suitable for businesses which regularly update their online store, and also has a complex ecommerce infrastructure with coordination of external services. Continuous changes are made to the catalog and the stock availability. Frequent campaigns and marketing content updates are likely to involve changes to multiple web pages. The business may also need to update the store configuration and deploy new code at least once a month, possibly for multiple countries. There is an observable trend of visits and sales during the day.
An unexpected change or a UI/CX error can cause high bounce / low conversion rates, also eventually affecting the brand reputation. It is acceptable from the business standpoint to be aware of and to escalate the issue in less than 6 hours.
This test frequency usually requires a dedicated team to monitor Valido at least 8 hours a day, 7 days a week.
Example 3: Test execution every hour (or greater) This test frequency meets the demand of enterprise businesses which update their online store at a high frequency, due to its complex infrastructure, interactions with external services and integrations with best-of-breed applications. Continuous changes to the catalog and the stock availability are made by dedicated teams. Frequent campaigns and marketing content updates occur on different pages, across different languages and country sites. Continuous integration and continuous delivery (CI/CD) is a common practice, with the likelihood of weekly code deploys. There is a recognizable pattern of visits and sales throughout the day and the various periods of the year.
An unexpected change or a visual error causes high bounce / low conversion rate. The business deems brand trustworthiness and reputation as a key focus of its business, therefore UI/CX errors should be escalated ASAP.
This test frequency requires a dedicated team to monitor Valido 24 hours a day, 7 days a week.
How can Valido help? When you create a new test in Valido, the frequency appears in the configuration options:
However, creating and executing the test alone is not sufficient to meet your visual testing goals. You will also need to properly review the changes when Valido detect them. If you are unable to manage it all on your own, we at Valido can help you take care of it. In fact, we have a dedicated 24 x 7 support team that can handle the configuration, execution, and monitoring of your visual tests.
Are you interested? Contact us.