A/B Testing

Hi, this is Tom from InnoCraft. The company of the makers of Matomo (Piwik) Analytics which is used by over 1 million websites and apps in over 150 countries.

For years we had those discussions within our team and with the bosses about which solution would convert better than another. Sometimes they would be on a Monday morning even before the first coffee or on a Friday afternoon just before the weekend: “Let’s go with a red color here”, “No, I’m sure blue will work much better” and a third person comes in “I’m sure a lighter blue color is better”. Truth is, you never know what really works the best unless you test it.

Have you had similar situations? Do you want to consistently increase your conversions and revenue? Then we have built just the right thing for you and we are sure it will end your discussions too and help you grow your business. When you create an A/B Test, you compare different wordings, looks, layouts or entirely different pages with each other and you get scientifically proven results which version works better. No longer any guessing when you build your app or website.

Unsurprisingly - A/B Testing is one of the most requested feature for Matomo and since we built and started using it, it really made a difference and changed the way we work, think and make our decisions. Shall we use “Buy now” or “Subscribe now”? Which background or product image works better? Now we simply try different versions, test them and keep the best working solution that actually increases our conversions. We even review previously made decisions and we got some shocking results: we were sometimes very wrong about what converts best. Did you know that in some tests changing a button color from green to red increased clicks by over 20%? I’d be curious to hear about your remarkable changes once you start testing.

A/B Testing is built on top of Matomo which means you get all those benefits and features from Matomo on top. Like data ownership, no data limits, being able to host it yourself on premise and use it in the intranet etc. That’s why Matomo is so popular among businesses, corporations and governments. Matomo is used and trusted by over a million websites and apps.

How it works

The A/B Testing plugin lets you easily create experiments and compare different versions of your website or app and will automatically detect which variation will increase your revenue or conversions. Experimenting and measuring the effect of each change you make with A/B testing gives you the confidence to always make changes which increase your revenue and conversions.

A/B tests are also known as experiments or split tests. In an A/B test you show two or more different variations to your users (visitors) and the variation that performs better wins. When a user enters the experiment, a variation will be randomly chosen and the user will see this variation for all subsequent visits. Matomo A/B testing uses advanced statistical analysis to detect which variation performs better for your conversion goals and success metrics.

Ultimately you can eliminate the guess work in your decision making and maximise your success!

Our promise

A/B Testing is simple to use, even if you have never created an A/B test before. It is hand-crafted by the makers of Matomo with top quality and we are convinced once you start using it, you will never want to miss it again which is why we give you a 100% money back guarantee for 14 days. We have had many delighted users that increased their revenue by more than the cost for this plugin with just one test. We have even had users that shared with us how they enjoy the moment when they tell their boss he or she was wrong because a test has proven his or her suggestion converts worse. So try it now and let us know how you do. We are happy to help you get started and to hear your story how it changes your life.

About InnoCraft

We at InnoCraft are the creators of Matomo and know it best. We help our clients get started, configure, monitor and make the most of their Matomo analytics service. We also offer unique analytics products and services that help grow your business and meet the needs of small, medium and large businesses alike.

Features

• The user interface guides you through all stages from creating an experiment to interpreting the results.
• Run an experiment in the browser or on a server using the JavaScript A/B testing framework.
  • All you need to do is to implement a method that defines what happens when a variation gets activated.
  • Possibility to test entirely different versions of your website by using redirects.
• Run an experiment on a server (PHP, C#, Python, Ruby, ...).
• Run an experiment in applications, games and on mobile devices like Android or iOS in combination with pretty much any A/B testing framework.
• Run an experiment to see how different search marketing campaigns, ad campaigns or email marketing campaigns influence the browsing behaviour on your website.

Reporting features

• Once an experiment is running:
  • Shows the description, hypothesis and goal of the experiment in the report of the experiment.
  • Shows an evolution of your success metrics for each variation.
  • Shows how the different variations compared between each other, for each of your success metrics.
  • Shows notifications for each success metric once a potential winner or loser is detected.
  • Makes sure the results are statistically significant and enough visitors or users have entered the experiment.
• Segment A/B testing reports by any existing Matomo segments.
• Segment Matomo reports by experiments and variations.

Manage A/B tests

• Users with admin access can create unlimited A/B tests (experiments).
• Assign as many variations to your experiment as you like.
• Define success metrics like:
  • Goal conversions,
  • Goal revenue,
  • Ecommerce orders,
  • Ecommerce order revenue,
  • Total conversions,
  • Total revenue,
  • Page views,
  • Bounces,
  • Visit length (time).
• Define which improvement rate you expect from an experiment.
• Define the Confidence Threshold you need to be statistically sure the change is not due to randomness.
• Define on which pages an experiment should (or should not) be activated.
• Define how much traffic the experiment should get in total.
• Allocate a different amount of traffic to your variations.
• Schedule your experiments to run from a start date and/or to finish on a given date.
• Integrates with the Activity log plugin to let you monitor and audit changes to your A/B tests.

Export and API features

• HTTP API to create, update, start and finish experiments.
• HTTP API to fetch all A/B testing reports and metrics overall and for each variation.
• Get access to all the raw data via MySQL for 100% data ownership.

Languages

A/B Testing is translated into the following languages:

• English
• German (Deutsch)
• French on request

More information

To learn more about the plugin have a look at the A/B Testing website, A/B Testing User Guide, A/B Testing FAQ and the developer documentation.