Experiments allow you to determine if changes you make to a page component lead to more visitor conversions to a specific goal. You can experiment with the content a component displays and with the audience a component is targeted to. A running experiment serves visitors one of the variants of a page component, tracks conversions for each variant and optimizes variant serving based on visitor analysis. After the experiment has been running for a while you can complete the experiment. Usually you complete an experiment if one variant is a clear winner, i.e. realizes a significantly higher conversion than the other variants.
An experiment tests whether a change to a page component's configuration - creating a variant of the page component - leads to better conversion. Multiple variants can be tested in the same experiment. An experiment is either a content experiment or an audience experiment.
A content experiment tests whether a variation in a page component's configuration (i.e. which content it contains, and how it is presented) leads to better conversion, compared to the current default configuration. A content experiment is similar to a traditional A/B test, although Hippo allows testing of more than two variants and uses a different, more refined algorithm.
An audience experiment tests whether a page component leads to better conversion when it is targeted at a specific audience, compared to when it is not.
A page component is a part of a web page containing functionality which can be configured in the CMS Channel Manager.
A variant is an alternative configuration of a page component. It either has different configuration parameters such as the content it displays (in case of a content experiment), or is targeted at a different (specific) audience (in case of an audience experiment).
A page goal is defined as a visitor reaching a pre-determined page after viewing the page the experiment runs on. A goal determines when a conversion takes place.
A conversion takes place when a visitor reaches a pre-determined goal.
The winner of an experiment is the variant tested by the experiment that leads to most conversions.
How do experiments work?
- In one experiment:
- you can have only one goal.
- you can test from 2 up to 26 variants of a single component.
- you can test any number of components on the same page.
- you can't add any more variants or components while the experiment is running.
- On one page:
- you can run multiple experiments (each with a different goal).
- you can run an experiment with the same goal as a different experiment on a different page.
- The chance a visitor is served a particular variant:
- is equal for each variant at the start of an experiment, when no data has been gathered yet.
- converges towards the 'winning' variant as the experiment progresses and more data on conversions is gathered.