The A/B Test Basic Evaluation presents a high-level overview of the results of your A/B test. It lets you compare the performance of your campaign compared to a control group or a different version and see whether there are any uplifts in the key metrics.

## This is how your analysis will look:

1155


## You will get the following insights from this report:

Column Title
Column Title
**Visitors**The number of people who have been targeted by the campaign.
**Customers** The number of people who have made a purchase after having been targeted, and within a predefined time frame.
**Conversion** = Customers/Visitors
**Average Order Value (AOV)**= Revenue/Customers
**Revenue per Visitor (RPV)** = Revenue/Visitors
**Revenue**The amount of revenue generated by the campaign. This metric uses a predefined time frame that starts at the moment the customer has been targeted.

# How to create the A/B test evaluation in Bloomreach Engagement

In this guide, we will show you how to create a report for a campaign showing a so-called "View count [weblayer](🔗)". We will look at the purchase behavior of visitors within 24 hours after they saw the weblayer, and compare it to the control group within the same time period.



Note that you can also use this logic to create evaluations for any campaign: experiments, emails, recommendations, etc.

This evaluation has 3 parts that will be explained in this guide:

  • A basic [segmentation](🔗) of customers

  • An [expression](🔗)

  • The final [report](🔗) (which makes use of the segmentation and expression)

RequirementsColumn Title
**Bloomreach Engagement skills**Basic - we will show you step-by-step how to create every part of this evaluation. We assume you can navigate and do basic operations in Bloomreach Engagement.
**Data and tracking**_ Tracking of the `purchase` event _ Tracking the appropriate event for the campaign you want to evaluate (these are tracked automatically) _ `show` for weblayers and experiments _ `campaign > delivered` for evaluating emails

## Creating the Segmentation

We need this segmentation to divide the customers into 3 groups:

  • **Mix** - customers that saw the weblayer but also fell into the control group. See the [FAQ](🔗) to understand why this happens.

  • **Variant A** - customers who were shown the weblayer

  • **Control Group** - customers who didn't see anything

(Script tags will be stripped)

1600


**Getting the `banner_id`** You can get this value when you go to your weblayer and copy the following value from the URL. You could also use the `banner_name` here. However, if you have more weblayer campaigns under this name, your analysis will be invalidated. The ID is always unique, that's why it is a good practice to use it.

1946

(Script tags will be stripped)


It is important to have the "Mixed" segment as the first one on the left side. Read the documentation about [segmentations](🔗) for explanation.

1600


Good job!

Save your work. You have now created the customer segmentation with 3 segments: Variant A, Control Group, and Mixed. This means you can now easily filter your customers depending on which segment they belong to and use this in your other analyses.

## Creating the Expression

This [expression](🔗) will calculate (using a [running aggregate](🔗)) for each event `purchase` the difference in seconds between that event and the last time the buyer saw our weblayer. This will enable us to include time attribution in the evaluation.

(Script tags will be stripped)

1600

(Script tags will be stripped)

1280


Good job!

Save your work. You can now move on to the last step - creating the final report using both the segmentation and the expression.

## Creating the Report

This will be the overall structure of your report. The next steps will guide you how to create each part of it.

Column Title
Column Title
**ROWS**Viewcount Groups (that's the segmentation you created before)
**COLUMNS**n/a
**METRICS**You will define 6 metrics: _ Visitors _ Customers _ Conversion _ AOV _ RPV _ Revenue
**CUSTOMER FILTER**A global filter that will exclude the "Mixed" group from the evaluation.
**EVENT FILTER**A global filter that will narrow down the metrics so they only relate to our weblayer campaign. It will also define the time attribution window.
(Script tags will be stripped)

1600

(Script tags will be stripped)

1280


Metric - Visitors Event filter specifying the banner

(Script tags will be stripped)


Note that every time you create a new metric, it is easier to copy the previous metric and then just edit it.

1792


Metric - Customers

(Script tags will be stripped)

1037


Now the global event filter is complete.

(Script tags will be stripped)

1036

(Script tags will be stripped)

1092

(Script tags will be stripped)

1804

(Script tags will be stripped)

1600

(Script tags will be stripped)

1370


Amazing!

You have now created a basic A/B test evaluation. This is one of the fundamental reports used heavily by our consultants and we are sure you will use it a lot as well.

Advanced tip

If you want to get the absolute uplift (how much money the campaign made), create a metric using this formula: (RPV Variant A - RPV Control Group) x Visitors Variant A.

# Troubleshooting / FAQ

See our [troubleshooting guide](🔗) to find answers to the following questions and more:

  • Why do I have the "mixed" group in my customer segments?

  • I don't see the expression/segmentation/aggregate I created in the list of events/attributes