Optimization testing

Development model: MVC Portal Engine
✖
Copy to clipboard

Kentico EMS required

Features described on this page require the Kentico EMS license.

Making improvements on a website can be a difficult process, since it is often not possible to know ahead of time whether changes will have a positive effect, or which modification out of several options will bring the best results. You can resolve these issues by running optimization tests for pages.

Optimization testing allows you to create different versions of a page (or specific parts of a page) and to evaluate them according to the behavior of the website’s visitors. You can confirm which changes are actually beneficial and use the content that works best for the users who visit your website. The testing process does not interfere with browsing on the website. Visitors do not need to give any feedback manually.

There are two different techniques that you can use to optimize pages in Kentico. Each type of testing has its advantages and is intended for different scenarios:

A/B testing

  • Divides incoming traffic between two or more different variants of a page
  • Tracks the results for each page variant as a whole (measures the combined effect of all changes made to the page)
  • Each variant is a separate page in the content tree

Advantages of A/B testing

  • Easy way to test large-scale changes—small changes can be tested as well
  • Suitable for exploring the combined potential of all page changes
  • Can provide quick results as the number of variants is typically lower than that of MVT

Disadvantages of A/B testing

  • Impossible to detect the importance of individual page elements

Multivariate testing (MVT)

  • Allows you to make multiple individual modifications to the content of a single page
  • Tracks the results for each specific change made to the page

Advantages of MVT

  • Easy way to test isolated changes to page elements
  • Suitable for fine-tuning a page
  • Can test many combinations of elements on a page

Disadvantages of MVT

  • May require more time or traffic to get a similar degree of statistical significance as A/B testing

Should I start off with A/B or MVT?

It is recommended to start optimizing your pages on a larger scale, using A/B testing. Once you choose the best page using A/B testing, you can use MVT to fine tune individual elements of the page.

Note that running both A/B testing and MVT on the same pages at the same time is not supported.

Both types of optimization testing measure results by tracking the activity of users after they access the tested page and view one of the different content versions. Actions that are desired from visitors are represented in the system as Conversions. Typical examples of conversions are product orders, registrations, newsletter subscriptions, views of special pages etc. When a user performs the action tracked by a conversion, the system logs a Conversion hit.

Testing is most recommended for key sections of the website that receive the most traffic, such as the default home page (landing page).