With most savvy e-commerce brands now using UX analytics solutions to augment their testing tools to determine what areas are most in need of improvement on their websites, it raises the question: just how much time and money is being wasted by A/B testing alone without UX insights? In this piece for RetailTechNews, Duncan Keene (pictured below), UK MD, ContentSquare, explores whether the end of A/B testing is nigh.

More testing isn’t always best

The vast majority of e-commerce teams still waste more time than may be needed testing absolutely everything when, instead, they would be better off prioritising the tests they’re running to only implement those they are confident of a win from, or those tests which will make the greatest impact on improving conversion or the user experience. But why are e-commerce teams still investing time, money, and effort on A/B testing? If it’s a case of "this is what we’ve always done” or “because my manager told me to” then they really should stop – right now.

A/B testing can be a costly investment, if you are not using it wisely. For example, do not A/B test a broken link – just fix it! This may seem like obvious advice, as fixing the broken links on sites and checking 404 logs regularly should be standard practice by now.

So, what is needed to truly innovate? Simply for e-commerce teams to take the next step and not just find and fix the broken sections and links on a site, but also identify which areas on the site have recurring problems that may be annoying visitors. But traditional analytics can only tell e-commerce teams what customers are doing on their site, not why and how they're doing it.

Traditional tools can also require analyst and IT teams to spend weeks delving into all of the information gathered, in order to find the specific answers that the e-commerce team needs to make their decision.

But, as most e-commerce, sales, and marketing teams will understand, time is money. Brands should instead look for tools that display aggregated user journeys visually, enabling them to understand why customers are leaving their site, as well as measure the revenue and behavioural contribution of any ‘block’ of content. There’s a huge need to understand golden or broken customer journeys, feed actionable insights to test hypotheses, and recognise why tests are winning or inconclusive.

The limits of traditional analytics

Many e-commerce teams are heavily reliant on tools like session replay to try and understand behaviour, but similar frustrations always arise. While session replay and heatmapping tools are used by e-commerce teams to see how visitors are interacting at a page-level, they are often cumbersome and difficult to extract specific and actionable data from. Session replays may reveal individual sessions in great detail, but they also require analysts to spend time getting an aggregated view of customer behaviour that testing activity can be based on, whereas UX analytics tools offer that from the start.

Duncan Keene, UK MD, ContentSquare

Heatmapping is a colourful way of visually identifying which areas of a site visitors are interacting with, by tracking scrolls, clicks, and how they move around, but it does not offer the comprehensive overview that UX analytics do. Although heatmapping can give you an idea of which website areas have the highest activity, UX analytics offers further insights, such as how many times a visitor has clicked on a link, where users are hovering without clicking, and even if they are clicking on an unclickable section.

UX analytics tools can help identify these annoyance areas and can be leveraged by e-commerce teams to incorporate data-driven design – which finds out where users are leaving a site and why – to optimise the customer journey to reduce exit rates, and revise and tailor content to suit the users.

How are UX tools helping to identify user frustration?

Rather than only running tests that they are confident of winning, e-commerce teams who combine a UX-analytics solution with their existing testing methods are in a better position to identify the root cause of a problem and prioritise a test to fix it. E-commerce teams are then in a better position to generate better test hypotheses and prioritise those hypotheses.

E-commerce teams are now able to determine what is most likely to work before testing, meaning they can run fewer, more targeted tests – both to solve known issues and to optimise conversion rate. Tests can be based on real data on user behaviour, rather on best practice or what competitors are doing.

Is this the end of A/B testing?

No, but there is now a new need for optimisation, brought about by UX analytics, that offers e-commerce teams stronger and more powerful testing and personalisation results by combining the two.

French online buying and selling platform PriceMinister Club revealed last year that 'less is more' and that instead of doing 100 small tests – with little to no impact of each – they wanted to do 10 tests instead, each with a higher ROI. Partnering with ContentSquare, PriceMinster discovered that there was a simple yet overlooked design issue with its password field, which was identified using the ‘click recurrence’ metric on the form page. A small design tweak to the login rectified any potential problems and had a significant impact on successful logins.

With the fast-paced round of technology innovations being released every year, retail brands need to better prioritise their finite resources and stop wasting time spent on traditional A/B testing and empower their testing roadmap with new tools like UX analytics.