Contextual personalization - a feature that drives your conversion rates and provides truly personalized content for your customers. It automatically assesses all the chosen contexts of your customers and utilizes the insights immediately to serve the most performant variant to customers based on their specific context.

Difference between Contextual Personalization and A/B Testing

A/B testing for conversion rate optimization is limited - only 1 in 8 A/B tests drive significant change. And only 33% of marketers regularly use A/B or multivariate testing.
While the A/B test serves you the overall best variant of your tested campaign based on average performance, contextual personalization serves you what is best for each of your customers based on the contexts in which they are found.

Example:

• 70% of customers are interested in discounts, and 30% in free shipping. You might find out that the winning variant is a weblayer about a sale. Once the testing is finished, it is decided to use that variant for the entire customer base. However, by doing that, you just misunderstood 30% of your customers and showed them something they did not care about. Moreover, you missed an opportunity to direct them towards products from sustainable materials, which might have a substantially higher profit margin. Here, contextual personalization comes in - based on all the different contexts, it shows the best-fitting variant for each individual customer.

Instead of “which variant is best for everyone,” ask, “which variant is best for each customer?”

Case study - Increase your conversion rate!

The case study of bimago is an example of how you can increase your conversion rate using contextual personalization compared to standard A/B testing. Deploying personalized banners to bimago's customers led to a 44% increase in the conversion rate of subscriptions with test results comparing contextual personalization to standard A/B testing. Find out more about this case study here!

Terminology

• Variant/action - options to choose by contextual personalization from a set of maximum of 10 options.
• Reinforcement learning - part of the Artificial Intelligence area, where the agent/program is learning by constantly interacting with an environment with the goal to maximize the reward.
• Context - specific situation of each customer, such as the device or browser they use, in which country they are located, at what time of the day they are browsing, and many others.
• Contextual bandit/personalization - algorithm used in the background; it belongs to a broad range of reinforcement learning approaches.
• Feature - input variable used by the prediction model to make a prediction. An example of the feature is the customer's country or their number of purchases in the last year.
• Reward - information that is used by the model to learn and adapt for changes.

How exactly it works

The task for contextual personalization is to select the best possible variant of a weblayer per visitor. It analyses the context of every visitor, such as device, daytime, history, brand affinity, and others. It tries all variants of the weblayer for different contexts. Then it learns what works for what context and uses this knowledge to serve the best variant for the customer. However, even after that, contextual personalization continues to learn as those contexts and preferences of customers change over time.

You need to create several versions of a weblayer, define the optimization target, and set the list of features. Once you start the weblayer, the algorithm based on the chosen features then identifies the context for every customer and selects the best option of your predefined weblayers. That specific personalized weblayer is then shown to your customers. In the beginning, all the variants are served equally, but as more rewards are tracked for a specific context and variant, the winning variant for every context prevails.

🚧

Contextual personalization never serves the control group variant.

Setting up the Use Case

Watch this video to see a simple guide on how to set up a contextual personalization weblayer:

🚧

Disclaimer

This video shows an incorrect consent category. Use a consent category you normally use for personalized weblayers.

In order to create the most useful features, it is necessary to consider the following:

Make features dense.

Dense features are features that contain just a few categories. Such a feature is Customer type. This feature has categories, for example, 'Anonymous' and 'Registered' customers, each containing thousands of customers. Using this information helps contextual personalization to use the information learned from one 'Registered' customer to another 'Registered' customer.

The opposite of this approach is to create very sparse features such as 'Product title', which is unique for each product, making it hard for contextual personalization to reuse this information across different products.

📘

There should always be a way to make features denser. For example, instead of setting hours of the day as input, you could create three groups - morning, afternoon, and evening.

Use lower number but relevant features instead of hundreds of irrelevant features.

The more features you include, the more contexts are found, and the longer it will take until the contextual personalization learns what works and will utilize it. Therefore, use business knowledge and intuition to think about features that could help contextual personalization to make the right decision.

Example:

• Imagine you have three features. First is a device with three distinct values - smartphone, tablet, and laptop. Second is an OS with three distinct values - iOS, Android, and Windows. And the third feature is an RFM segment with ten distinct values. Calculating these values gives us the formula: 3 device values 3 OS values 10 RFM values. This makes 3_3_10 = 90. Thus, you have 90 different contexts, and the best variant for each context has to be found. It is done by understanding what best works for each of those contexts.

📘

Rule of thumb

Multiply 100 by the number of contexts and the number of variants. Then divide it by the estimated conversion rate (how many times the displayed weblayer is clicked). The result is the needed size of your audience.

Example:

• 100 x 20 contexts x 2 variants / 0.02 conversion rate = 200 000.
• If your expected uplift in conversions is about 20%, this result, 200 000 customers, is the approximate audience size of your weblayer needed to evaluate the contextual personalization with statistical significance. It also depends on the significance evaluation method you use and the level of confidence you seek.

Some features to be used

• User context: Device (Apple, Android, other), Browser (Safari, Chrome, other), OS (iOS, Android, Windows, other), Screen size (small, medium, big), Weather (sunny, rainy, other)
• General context: Time of the day (morning, afternoon, evening, night), Day of the week (workday, weekend), Country/region (from which country the customer is browsing), Size of the city (> 1M, >0.5M, <0.5M)
• User aggregates and segmentations: Number of purchases in total (>1, 1, 0), Number of engagements in the last 3 months (>100, >10, <10 (based on project analytics)), RFM segments, number of viewed products, Most commonly visited brand/category, Price preference (Visiting cheap/expensive, Ordering PLP based on price from the cheapest), Visiting mainly products in sale, Purchase probability segment (high, medium, low)

Best practices to create reward

• Establish what success for you is. Then you can optimize - if the goal is to subscribe to a newsletter, then optimize for subscription clicks.
• Consider immediate feedback. Even if your goal is to sell a product when showing the best-promoted products to a customer, do not optimize for purchases, as there might not be enough data points. Instead, a great way to show the contextual personalization that the right decision was made is by monitoring clicks on the weblayer.

Use Case examples

Here you can find three possible Use Cases:

1. Consider an example of 4 weblayers with different discount types:

• Free delivery (you can read more about this Use Case here)
• 10% off the first purchase
• 3€ discount
• Free gift

2. Or select which image or product you want to show first in a carousel.
3. Or select the best-performing homepage banner for each customer.
4. Select the best recommendation engine for each website visitor (e.g., choose from ‘Personalized for you’, ‘Top sellers’ or ‘New arrivals’ on the homepage)

However, there are many other possible Use Cases for your business.

The first example, a subscription banner with two discount types, will be used in this guide.

Step-by-step guide

1. Select a Use Case

Select a Use Case where personalization plays a relevant role in achieving desired results.

2. Create a weblayer

Create different variants of a weblayer and set all the relevant settings. Read more here.

3. Set the contextual personalization

In order to set the contextual personalization, go to the A/B test, choose Contextual personalization in the field indicated in the first picture, and then set the Traffic distribution.

Traffic split

As not only using contextual personalization but also evaluating the uplift against simple variants is essential, you need to send some customer traffic to a Comparative A/B test. A good ratio to set is sending 80% of the traffic to the contextual personalization and 20% to a Comparative A/B test. This way, the contextual personalization can learn quickly enough, and yet, relevant traffic goes to simple variants to be able to evaluate the gain with statistical significance.

Optimization target

It is necessary to let the contextual personalization model know once the served variants led to success for this model to work. You can do this automatically by setting this optimization target or manually utilizing the open API. However, remember that the automatic way works only for success that is tracked on the same page as the weblayer is displayed. For deeper tracking of the success in the funnel, using API is necessary.
In this case, you can do so via setting it to subscription banner click and specifying the banner_name, such as ‘New Subscription Banner’ or banner_id, which can be found in the URL.

Reward settings

In situations where the weblayer variants and their success have different business values (e.g., subscription with and without discount), you can promote this information to the model. In this case, however, it is not needed.

Features

Contextual personalization works with contexts, and the contexts are defined via selected features. As described above, the features can be anything from the visitor's device to the most visited product category to some behavior segment like RFM.

Settings

Under the Settings page, choose relevant data, including the Consent category.

Contextual personalization can serve different variants to the same visitors based on the current visitor context and learnings. If you do not wish this to happen, ensure that the weblayer is shown only once.

After setting up the contextual personalization, you are ready to run the weblayer.

Evaluation

To evaluate the uplift of contextual personalization against simple variants, a new banner event property is tracked - variant_origin.

Since contextual personalization and the comparative A/B test serve all the different variants, this property tells us the origin of the variant.

The best way to approach the evaluation is to create a segmentation with three segments: Contextual personalization / Variant A / Variant B.

It is expected that the performance of contextual personalization will outperform not only the whole comparative A/B test but also each variant separately. If not, serving the sole winning variant to every visitor will be more helpful.

There can be some reasons why contextual personalization would lose against the simple variants. Most often, it is the combination of the fact that the behavior of visitors cannot be predicted via the selected features and the fact that contextual personalization and the underlying algorithm, as part of the never-ending learning process, always serve several percent of the traffic randomly.

For example, in a simple case where one variant is better for every context, the worse variant will also be served via contextual personalization. Overall, they will lose against the single variant.

Post evaluation settings

The last step after the evaluation is to take action based on its results. If you are happy with the results and do not plan any adaptations and re-running the test, you need to select the winner - either the contextual personalization or one of the variants. This can be quickly done in the Contextual personalization settings in the Finish A/B test section.

🚧

This section is available only for running weblayers with non-100% traffic sent to the contextual personalization.

❗️

A confirmation window is displayed as this action cannot be undone after selecting the winner and clicking the Finish A/B Test button.

After the confirmation, all the traffic is served with the selected variant, and the Finish A/B test section informs about this.

Advanced settings

Reward settings

You may encounter situations where not all the variants of the same weblayer and the intended activity have the same business value. Imagine a subscription banner where one version offers a 10% discount for the next purchase and the second one does not. You can calculate how much business value the subscription without the discount has for your business. Not only directly via the discount but also in a long-term perspective - subscribers without discounts usually have higher customer lifetime value in the long run.

In situations like this, the difference between business values can be propagated to contextual personalization via Reward settings. If the value from the no-discount subscribers is twice as big as the value of discount subscribers, just set the reward for the no-discount version to 2 and the discount version to 1. What matters is the ratio between the numbers, not the absolute value itself (2:1 has the same meaning as 50:25). This also needs to be considered in the evaluation, as achieving twice the big conversion rate with the discount variant has the same business value.

📘

Use incremental reward as the customer is moving closer to the final goal.
For example: product view > add to cart > purchase

Optimization target

The optimization target is the action that triggers sending the reward to contextual personalization. This target is turned on by default in the contextual personalization Settings. This works with events tracked on the same page as the weblayer is displayed. If you need to track the reward later or are not on the same page, turn this option off and use the API to track the reward whenever needed. Make sure it is no later than 15 minutes, as this is the maximum delay between serving the weblayer and tracking the reward. Keep in mind that the bigger the conversion rate from impression to this target is, the faster it will take the contextual personalization model to learn relevant insights and use them.

Make sure that you have the reward_token stored to be able to send the reward. This can be done directly in the JavaScript of the weblayer. The reward token is in the object data.contextual_personalization.reward_token, so you can track it into the banner event or create a new dedicated event that can be easily expired just after the 15 minutes.

The data.contextual_personalization is present only for variants served by the contextual personalization. Therefore, do not try to get reward for variants served by the comparative A/B test in cases where you use it.

Sending the reward token can be easily done via a simple on_event triggered scenario:

In this example, after every purchase, a verification of whether the user was impressed by the contextual personalization is done. If yes, the reward is tracked.