A/B Testing for CRO

CRO is an acronym used by marketers referring to conversion rate optimisation – the process of increasing the number of conversions a website receives. There are several types of conversions a business could be aiming to increase; transactional sales, form completions, email signups or the number of visitors creating an account. These are all dependent on the business’s goals.

In order to optimise your website’s conversion rate, you need to understand what your customers’ needs and wants are when visiting your website. Simply asking your customers through surveys doesn’t guarantee any quality feedback that you could rely on. Instead, taking the data-driven approach using analytical tools, such as Google Analytics and Hotjar, enables you to physically see what is and isn’t working, ultimately preventing your conversion rate from increasing.

What Is A/B Testing?

CRO and A/B testing typically go hand in hand, as optimising a website’s conversion rate requires various testing to be carried out. A/B testing is simply the act of testing two or more variations of pages, content or elements, to help pinpoint what contributes to customers successfully being pushed down the marketing funnel and converting. There’s no correct “one size fits all” way for your website to appear and function to be successful; what works for one company’s website may not work for another. The specific data collected from your visitors is key to understanding where any metaphorical “roadblocks” are along the customer’s journey, enabling you to eliminate them through the process of A/B testing.

But What Data Should I Be Tracking?

Within Google Analytics, the ‘pages per session’ and ‘average session duration’ metrics are good to be aware of but need to be approached with discernment. Having a high average session duration could suggest customers are having to spend more time trying to find what they want which leads to frustration and abandoning the site. To ensure this isn’t the case, view and examine your analytics – when visitors are viewing lots of pages, are they resulting in conversions? If not, it might be an indication that there are too many roadblocks on their way to converting.

AB testing CRO

The two metrics which are essential to track are Bounce Rate and Conversion Rate. If there is a high Bounce Rate on a certain page, you can take that as a good indicator of where to start A/B testing. Once you identify the problem and fix it, your Bounce Rate on this specific page is likely to decrease, resulting in a spike in conversions. After all, Conversion Rate Optimisation (CRO) leads to more completed purchases or sign-ups, depending on the goal, rather than just increasing the traffic that comes into the site.

How Does A/B Testing Work?

Once decided on an A/B testing tool, some of the best being Optimizley, VWO, Omniconvert and Google Optimise, a code needs to be placed into the website in order to gain full control of the site and users’ experience. Now, using any of the website analytical tools previously mentioned, analyse where your visitors appear to be dropping off within your website. Identifying that helps narrow down what pages need reviewing and testing, allowing you to start building the new variations to test against within the A/B testing tool. Having conducted your research prior and identified where your visitors are dropping off, you can build the variations within the A/B testing tool, creating as little as one to as many variations desired.


If conducting multiple A/B tests simultaneously, ensure none of them conflicts with one and other. Avoid testing anything which can be viewed at the same time, as although some testing tools do limit each visitor to only see one campaign at a time, others however don’t. Consequently resulting in both test results being contaminated by one and other. So in general it’s good practice to keep the tests separate and prioritise one over the other initially.

The A/B testing tool allows you to decide what percentage of your site traffic are recipients of the test. For example, 100% traffic would equal 50% of visitors seeing the original site, also referred to as ‘the control’, the other half seeing the variant. The delivery is completely random, ensuring there are no biased results depending on external factors.

Before launching the test, there will be an option of setting goals for your tests. This is dependent on what you are trying to achieve. Either increasing conversion rate or something more specific like multiplying click-throughs of a button or being directed to the next stage of the funnel.

Browser cookies track and save whichever variation visitors land on. Though, if a visitor were to delete the cookies’ data before returning to the website or open it on another device, they could potentially view the control or another variation.

Each test should last for a minimum of 1 to 2 weeks, to ensure enough data has been collected and the results have normalised, clearly stating a ‘winner’. Keep in mind that the duration of your test will vary depending on your site’s average number of visitors, especially if you have multiple variations. Low traffic means the test needs to run longer to collect enough data and vice versa. The successful variation needs to be implemented for 100% of traffic until it’s been permanently built into the website.

Marketing Funnels & CRO Strategies

When it comes to testing, you’re able to change basically anything that already exists on site which your customers can see. There are some limits though, you’re unable to add net-new pages without adding them to the original website. For instance, if the homepage needed to be completely redesigned, you couldn’t test this without adding the new page variation to the original website. However, using your A/B testing tool, you could redirect a proportion of your visitors to the new page and the others to the original. The other limitation to testing includes the site functions which don’t already exist in the original code. For example, if you wanted to test an ‘Add to wishlist’ function, it would need to be written into the code first, but then be suppressed for a percentage of traffic.

You could test:

  • Images or Videos
  • Copy
  • Call to actions
  • Buttons
  • Pop-ups
  • Navigation
  • Forms

If you are just starting out with A/B testing then begin with features that everyone can see! Landing pages typically receive the most traffic, so are a great place to start pushing those new visitors down the funnel and further into your site.

Further A/B Testing Ideas

If your website data isn’t showing you the results you were hoping for in terms of conversions but are unsure of where the issue lies, here are some testing suggestions:

  • Perfect your checkout – Your visitors are on the home stretch, ready to make a purchase. The checkout should have as few distractions as possible, to ensure your customers make that transaction. Test removing irrelevant content which could persuade the customer to go backwards in the funnel. Display shipping costs upfront as this can be a surprise expense which sometimes discourages the user from continuing. Simplify navigation, so there’s no confusion. Remember to continue showing any price discounts, original price and the new price. This will remind the customer how much they are saving which will motivate conversions.
  • Experiment with major elements – Large blocks of content can be overwhelming so try hiding parts or breaking them up into smaller pieces. Show/hide banners or images which are pushing important content further down the page. Cyber security logos may be off-putting for customers, so test which works for you.
  • Auto sort by popularity – Try sorting your product categories by popularity. This may improve the chances of new visitors finding what they are looking for quicker.
  • Create clarity – Test any call to actions (CTA’s) that have a lot of text vs a short snappy one. Chances are the short one will generate more leads. Any ad copy, headlines or CTAs should be clear and simple to drive a response.

A/B Testing for CRO Results: How to Identify the Winner?

As mentioned earlier, each test should last at least 2 weeks. Your A/B testing tool will have a reports page that provides key information which contributes to deciding when the test should end and what variation won. There will be a metric called ‘confidence rate’ or ‘chance to beat control’, providing a percentage to each variation whether it’s predicted to be more successful than the original ‘Control’. The percentage has to be above 50%, otherwise, it’s a clear indication it has lost. To consider using any of the variations, they need to be at least 90%. Meaning there is little risk of the variation becoming a failure.

Next, a graph will present all the results. Ignoring the first few days, each line should have flatlined and stayed like that for 10 days or so to be able to stop the test.

Combining each of those points plus a sufficient amount of traffic will indicate a ‘winner’ of the test. If the confidence percentage is good but does not receive enough traffic, then let the test run longer, in order to obtain a fair result.

No Clear A/B Test Winners?

Some tests may be trial and error, not all will result in the variant being crowned the ‘winner’ but that doesn’t mean the test was unsuccessful. You have gained more information about your visitors’ browsing habits, which will help guide future tests. But how can you utilise this information to your benefit now? The best way is to analyse the data and create user personas to paint a clear picture of your target audience, which can not only guide your further CRO efforts but also inform your social media posts or email marketing campaigns.

Building A User Persona

Forming one primary persona and perhaps a couple of secondaries from analysing previous reports enables you to build specific segments within your analytical tracking tool to identify what website elements the target audience is using and where A/B testing for CRO should be implemented. To give an example, using demographics within your analytical tool, enables you to identify both the age and gender of your key persona. At this point, start to build a new segment within the tool including that information. Take a look at your customer interests and narrow down what your primary persona interests are, then add those to the segment too. Also, key information to incorporate is ‘Geo’. Your key persona’s language and geographic location are valuable in order to target specific regions for future campaigns. You can make the persona segment as specific or vague as desired, depending on your website traffic volume or business goals. Now having built the segment, you’re able to see the pages and elements which are the driving factors of your key persona making a conversion. However, more importantly, you’re able to recognise where they are dropping off within your site which pinpoints exactly what to A/B test to optimise your website for your key persona.

How Many A/B Tests Do I Need to Run To See an Increase in Conversions?

The first test won’t magically increase your conversion rate overnight. A/B testing for CRO is an iterative process of removing both big and small issues throughout your conversion funnel, creating a seamless experience for customers. Removing a stumbling block in the middle of the funnel won’t automatically make the entire journey clear too. It will take several improvements throughout the funnel until you start seeing your conversion rate increase.

It’s vital to regularly track your website’s data through analytical tools, in order to identify more testing opportunities. Through constant tracking, you will begin to spot behavioural patterns within your data. Completing one test and observing the subsequent data could potentially reveal another page which is now in need of testing. That new issue that arose as a result of a recent improvement proves how necessary observing your website’s data is at every step of the testing process. For example, improving an element on one page and successfully increasing the click-through rate will inevitably lead to more traffic being directed through to the next page. Consequently, the following page to which you have successfully boosted traffic levels, may not have previously received sufficient data to identify any testing opportunities. However, as a result, you may now be experiencing a high drop off rate, clearly showing the need for this case to be tested and optimised to increase conversions.

CRO Strategies

Here at GLO we continually analyse the data we are tracking to spot patterns and testing opportunities, which in turn help us to optimise our website and offer a seamless experience for our visitors.

Common CRO Mistakes to Avoid

Now, if you are sitting there feeling slightly overwhelmed or confused after reading the entirety of that information, let’s take a step back and recap. Conversion Rate Optimisation is an ongoing process that involves tweaking elements of your website in order to improve user’s experience and as a result generate more conversions, whether those are sales, form submissions or email newsletter subscriptions. By working on your CRO you can generate more leads via your website as your visitors will come across fewer roadblocks.

One of the best ways to improve your CRO is through A/B testing – the process of testing new variations of pages or elements on a website against the original. To decide what to test, use a website analytical tool to figure out where most of your current traffic is dropping off or start with landing pages that typically receive the most traffic. Some of the metrics that may indicate areas or pages worth testing include the Bounce Rate and Conversion Rate. Run your tests for a minimum duration of one to two weeks and allow the results to normalise – you know which version is your winner when you see one with at least a 90% success rate.

And finally, don’t forget that not all A/B tests you do will have the goal of increasing your conversion rate, especially in the beginning. You could set the goal to measure the number of times a specific element that’s being tested is clicked and therefore pushing those users closer to making a conversion. Furthermore, not all tests will have a clear winner but the data you collect can still be utilised to build a primary persona. This data can help you to understand who your key customer is and constantly ensure any A/B tests are increasing the conversion rate within that segment.


Just like with any optimisation efforts, it is important to strike a good balance between not responding to the data and doing too much to try and fix the issues. Every visitor wants a simple and easy browsing experience. So when it comes to optimising or redesigning your website and completing A/B testing for CRO, here are some mistakes you should try to avoid:

  • Over personalisation – Personalised shopping experiences may be the latest trend but that doesn’t mean a customer’s experience should be 100% personalised. It can be off-putting sometimes for a customer to see actually how much information you know about them, resulting in potential losses. Test what performs best, do visitors want to see suggested items based on what they’ve already viewed or view new products that they may not have considered before?
  • Having multiple navigation methods – You want to ensure a streamlined experience. If adding new navigational methods, ensure it is easy for a customer to use or give hints on how to use new features.
  • Obsessing over colour – Of course, some colours are more eye-catching than others but overall it’s not an element to spend too much time worrying about it. Fix substantial problems before testing little details.
  • Carousels vs Static Images – Carousels that are especially featured on landing pages can bring the site to life and make it instantly more engaging, however, users typically click on the first slide, therefore missing the rest of the information. A/B testing will determine which works best for your users.
  • Don’t test without any data to back up your decision – This is a waste of time. Randomly testing elements to see if they work better is very time-consuming too. Figure out what content needs to be tested using your site’s data. You should A/B test new content before adding it to the original site in case it damages your conversion rate.
  • Focusing too much on the landing page – Yes, the landing page is very important! But if you know there are bigger issues further down the funnel, test and fix those first before driving more customers into the funnel.

Request a Callback

Are you already applying any of our A/B testing for CRO ideas on your website? Request a callback to get free advice about how you can improve your conversion rate.