A/B Testing

Hi, this is Tom from Matomo, the creators of the web analytics platform which is used by over 1.4 million websites and apps in over 190 countries.

For years we had those discussions within our team and with the bosses about which solution would convert better than another. Sometimes they would be on a Monday morning even before the first coffee or on a Friday afternoon just before the weekend: “Let’s go with a red color here”, “No, I’m sure blue will work much better” and a third person comes in “I’m sure a lighter blue color is better”.

Truth is, you never know what really works the best unless you test it.

Have you had similar situations? Do you want to consistently increase your conversions and revenue? Then we have built just the right thing for you and we are sure it will end your discussions too and help you grow your business. When you create an A/B Test, you compare different wordings, looks, layouts or entirely different pages with each other and you get scientifically proven results which version works better. No longer any guessing when you build your app or website.

Take the guesswork out of your website optimization and reduce the risk in all your decision making.

Unsurprisingly - A/B Testing is one of the most requested feature for Matomo and since we built and started using it, it really made a difference and changed the way we work, think and make our decisions. Shall we use “Buy now” or “Subscribe now”? Which background or product image works better? Now we simply try different versions, test them and keep the best working solution that actually increases our conversions. We even review previously made decisions and we got some shocking results: we were sometimes very wrong about what converts best. Did you know that in some tests changing a button color from green to red increased clicks by over 20%? I’d be curious to hear about your remarkable changes once you start testing.

Start testing and improving everything from aesthetics like content and colors to back-end functionality. Even small changes can have a huge impact in your conversions and revenue.

A/B Testing is built on top of Matomo which means you get all those benefits and features from Matomo on top. Like data ownership, no data limits, being able to host it yourself on premise and use it in the intranet etc. That’s why Matomo is so popular among businesses, corporations and governments. Matomo is used and trusted by over a million websites and apps.

How it works

When you have the A/B Testing plugin, it lets you easily create experiments and compare different versions of your website or app and it will automatically detect which variation increases your revenue or conversions.

Experimenting and measuring the effect of each change you make with A/B testing gives you the confidence to always make changes which increase your revenue and conversions.

A/B tests are also known as experiments or split tests. In an A/B test you show two or more different variations to your users (visitors) and the variation that performs better wins. When a user enters the experiment, a variation will be randomly chosen and the user will see this variation for all subsequent visits. Matomo A/B testing uses advanced statistical analysis to detect which variation performs better for your conversion goals and success metrics.

Ultimately you can eliminate the guess work in your decision making and maximise your success!

Our promise

A/B Testing is simple to use, even if you have never created an A/B test before. It is hand-crafted by the makers of Matomo with top quality and we are convinced once you start using it, you will never want to miss it again.

This is why we give you a free 30 day trial. We will remind you about the subscription seven days before the end of the trial. There are no strings attached.

We have had many delighted users that increased their revenue by more than the cost for this plugin with just one test. We have even had users that shared with us how they enjoy the moment when they tell their boss he or she was wrong because a test has proven his or her suggestion converts worse. So try it now and let us know how you do. We are happy to help you get started and to hear your story how it changes your life.

Perfectly integrated and supported

We are the creators of Matomo and know it better than anyone else. This means all plugins are perfectly integrated into Matomo and come with outstanding features and quality to grow your business. We help our clients get started, configure, monitor and make the most of their Matomo analytics service. We also offer unique analytics products and services that help grow your business and meet the needs of small, medium and large businesses alike.

Features

• The user interface guides you through all stages from creating an experiment to interpreting the results.
• Run an experiment in the browser or on a server using the JavaScript A/B testing framework.
  • All you need to do is to implement a method that defines what happens when a variation gets activated.
  • Possibility to test entirely different versions of your website by using redirects.
• Run an experiment on a server (PHP, C#, Python, Ruby, ...).
• Run an experiment in applications, games and on mobile devices like Android or iOS in combination with pretty much any A/B testing framework.
• Run an experiment to see how different search marketing campaigns, ad campaigns or email marketing campaigns influence the browsing behaviour on your website.
• Force the activation of a specific variation via a URL parameter so you can test it easily and share the URL with colleagues when you need to get each variation approved before running a test.

Reporting features

• Get an overview over all experiments in just one page.
• Once an experiment is running:
  • Shows the description, hypothesis and goal of the experiment in the report of the experiment.
  • Shows an evolution of your success metrics for each variation.
  • Shows how the different variations compared between each other, for each of your success metrics.
  • Shows notifications for each success metric once a potential winner or loser is detected.
  • Makes sure the results are statistically significant and enough visitors or users have entered the experiment.
• Segment A/B testing reports by any existing Matomo segments.
• Segment Matomo reports by experiments and variations.
• Access raw data for advanced custom analysis

Manage A/B tests

• Users with admin access can create unlimited A/B tests (experiments).
• Assign as many variations to your experiment as you like.
• Define success metrics like:
  • Goal conversions,
  • Goal revenue,
  • Ecommerce orders,
  • Ecommerce order revenue,
  • Total conversions,
  • Total revenue,
  • Page views,
  • Bounces,
  • Visit length (time).
• Define which improvement rate you expect from an experiment.
• Define the Confidence Threshold you need to be statistically sure the change is not due to randomness.
• Define on which pages an experiment should (or should not) be activated.
• Define how much traffic the experiment should get in total.
• Allocate a different amount of traffic to your variations.
• Schedule your experiments to run from a start date and/or to finish on a given date.
• Integrates with the Activity log plugin to let you monitor and audit changes to your A/B tests.

Export and API features

• HTTP API to create, update, start and finish experiments.
• HTTP API to fetch all A/B testing reports and metrics overall and for each variation.
• Get access to all the raw data via MySQL for 100% data ownership.

More information

To learn more about the plugin have a look at the A/B Testing website, A/B Testing User Guide, A/B Testing FAQ and the developer documentation.