This feature enables even deeper personalization for your customers. Contextual personalization automatically analyzes all the different contexts of your customers and utilizes the insights immediately.
While A/B test serves you the overall best variant of your tested campaign based on average performance, contextual personalization serves you what is best for each of your customers based on the contexts in which they are found.
The case study of bimago is an example of how you can increase your conversion rate using contextual personalization compared to standard A/B testing. Deploying personalized banners to bimago's customers led to a 44% increase in the conversion rate of subscriptions with test results comparing contextual personalization to standard A/B testing. Find out more about this case study here!
- Variant/action - options to choose by contextual personalization from a set of maximum of 10 options.
- Reinforcement learning - part of the Artificial Intelligence area, where the agent/program is learning by constantly interacting with an environment with the goal to maximize the reward.
- Context - specific situation of each customer, such as the device or browser they use, in which country they are located, at what time of the day they are browsing, and many others.
- Contextual bandit/personalization - algorithm used in the background; it belongs to a broad range of reinforcement learning approaches.
- Feature - input variable used by the prediction model to make a prediction. An example of the feature is the customer's country or their number of purchases in the last year.
- Reward - information that is used by the model to learn and adapt for changes.
Reinforcement learning is the base of contextual personalization. It means that different variants are tried for different contexts and the knowledge gained already after the first interaction is then applied to other customers. If the behavior of customers changes, contextual personalization adapts and re-learns as time is progressing.
You need to create several versions of a weblayer, define the optimization target, and set the list of features. Once you start the weblayer, the algorithm based on the chosen features then identifies the context for every customer and selects the best option of your predefined weblayers. That specific personalized weblayer is then shown to your customers. In the beginning, all the variants are served equally but as more rewards are tracked for specific context and variant, the winning variant for every context prevails.
Control group variant is never served by the contextual personalization.
Watch this video to see a simple guide on how to set up a contextual personalization weblayer:
In order to create the most effective features it is necessary to consider the following:
Make features dense.
Dense features are features that contain just a few categories. Such a feature is Customer type. This feature has categories, for example, Anonymous and Registered customers, each containing thousands of customers. Using this information helps contextual personalization to use the information learned from one Registered customer to another Registered customer. The opposite of this approach is to create very sparse features such as Product title, which is unique for each product, and it makes it hard for contextual personalization to reuse this information across different products.
There should always be a way to make features denser. For example, instead of setting hours of the day as an input, you could create 3 groups - morning, afternoon, and evening.
Use lower number but relevant features instead of hundreds of irrelevant features.
The more features you include the more contexts are found and, thus, the longer time it will take till the contextual personalization learns what works and will utilize it. Therefore, use business knowledge and intuition to think about features that could help contextual personalization to make the right decision.
Imagine you have 3 features. First is a device with 3 distinct values - smartphone, tablet, and laptop. Second is an OS with 3 distinct values - iOS, Android, and Windows. And the third feature is an RFM segment with 10 distinct values. Calculating these values gives us the following formula: 3 device values 3 OS values 10 RFM values. This makes 3310 = 90. Thus, you have 90 different contexts and the best variant for each context has to be found. It is done via understanding what works for each of those contexts the best.
Rule of thumb
Multiply 100 by the number of contexts and the number of variants. Then divide it by the estimated conversion rate (how many times the displayed weblayer is clicked). The result is the needed size of your audience.
100 x 20 contexts x 2 variants / 0.02 conversion rate = 200 000.
If your expected uplift in conversions is about 20%, this result, 200 000 customers, is the approximate audience size of your weblayer that is needed to be able to evaluate the contextual personalization with statistical significance. It also depends on the significance evaluation method you use and the level of confidence you seek.
Examples of features
- User context: Device (Apple, Android, other), Browser (Safari, Chrome, other), OS (iOS, Android, Windows, other), Screen size (small, medium, big), Weather (sunny, rainy, other)
- General context: Time of the day (morning, afternoon, evening, night), Day of the week (workday, weekend), Country/region (from which country the customer is browsing), Size of the city (> 1M, >0.5M, <0.5M)
- User aggregates and segmentations: Number of purchases in total (>1, 1, 0), Number of engagements in the last 3 months (>100, >10, <10 (based on project analytics)), RFM segments, number of viewed products, Most commonly visited brand/category, Price preference (Visiting cheap/expensive, Ordering PLP based on price from the cheapest), Visiting mostly products in sale, Purchase probability segment (high, medium, low)
Best practices to create reward
- Establish what is a success for you. Then you can optimize - if the goal is to subscribe to a newsletter then optimize for subscription clicks.
- Consider immediate feedback. Even if your goal is to sell a product when showing the best-promoted products to a customer, do not optimize for purchases as there might not be enough data points. Instead, a great way of showing the contextual personalization that the right decision was made is through monitoring clicks on the weblayer.
Here you can find three possible Use Cases:
- Consider an example of 4 weblayers with different discount types:
- Free delivery (you can read more about this Use Case here)
- 10% off the first purchase
- 3€ discount
- Free gift
- Or select which image or product you want to show first in a carousel.
- Or select the best-performing homepage banner for each customer.
However, there are many other possible Use Cases for your business.
In this guide, the first example, a subscription banner with two discount types, will be used.
Select a Use Case where personalization plays a relevant role to achieve desired results.
Create different variants of a weblayer and set all the relevant settings. Read more here.
In order to set the contextual personalization, go to the A/B test, choose contextual personalization in the field indicated in the first picture, and then set the Traffic distribution.
As not only using contextual personalization but also evaluating the uplift against simple variants is essential, you need to send some customer traffic to a Comparative A/B test. A good ratio to set is sending 80% of the traffic to the contextual personalization and 20% to a Comparative A/B test. This way, the contextual personalization is able to learn quickly enough, and yet, relevant traffic goes to simple variants in order to be able to evaluate the gain with statistical significance.
It is necessary to let the contextual personalization model know once the served variants led to success for this model to work. You can do this automatically via setting this optimization target, or manually via utilizing the open API. However, keep in mind that the automatic way works only for success that is tracked on the same page as the weblayer is displayed. For deeper tracking of the success in the funnel, using API is necessary.
In this case, you can do so via setting it to subscription banner click and specifying the banner_name, such as ‘New Subscription Banner’, or banner_id, which can be found in the URL.
In situations where the weblayer variants and their success have different business value (e.g. subscription with and without discount), you can promote this information to the model. In this case, however, it is not needed.
Contextual personalization works with contexts and the contexts are defined via selected features. As described above, the features can be anything from the visitor's device to the most visited product category, to some behavior segment like RFM.
Under the Settings page, choose your relevant data, including the Consent category.
Contextual personalization can serve different variants to the same visitors, based on the current visitor context and learnings. If you do not wish for this to happen, make sure that the weblayer is shown only once.
After setting up the contextual personalization, you are ready to run the weblayer.
To be able to evaluate the uplift of contextual personalization against simple variants a new banner event property is tracked - variant_origin.
Since contextual personalization and also the comparative AB test serve all the different variants, this property is telling us what was the origin of the variant.
The best way to approach the evaluation is to create a segmentation with 3 segments: Contextual personalization / Variant A / Variant B
It is expected that the performance of contextual personalization will outperform not only the whole comparative A/B test but also each variant separately. If not, then serving the winning sole variant to every visitor will be more useful.
There can be some reasons why contextual personalization would lose against the simple variants. Most often, it is the combination of the fact that the behavior of visitors cannot be predicted via the selected features and the fact that contextual personalization and the underlying algorithm, as part of the never-ending learning process, always serve several percent of the traffic randomly.
For example, in a simple case where one variant is better for every context, the worse variant will be also served via contextual personalization. Overall, they will lose against the single variant.
Post evaluation settings
The last step after the evaluation is to take action based on its results. If you are happy with the results, and you do not plan any adaptations and re-running the test, you just need to select the winner - either the contextual personalization or one of the variants. This can be easily done in the contextual personalization settings in the Finish A/B test section.
This section is available only for running weblayers with non-100% traffic sent to the contextual personalization.
As this action cannot be undone after selecting the winner and clicking the Finish A/B Test button a confirmation window is displayed.
After the confirmation, all the traffic is served with the selected variant and the Finish A/B test section informs about this.
You may encounter situations where not all the variants of the same webalyer and the intended activity have the same business value. Imagine a subscription banner where one version offers a 10% discount for the next purchase and the second one does not. You can calculate how much business value the subscription without the discount has for your business. Not only directly via using the discount, but also in a long-term perspective - subscribers without discounts usually have higher customer lifetime value in the long run.
In situations like this one, the difference between business value can be propagated to contextual personalization via the reward settings. If the value from the no-discount subscribers is twice as big as the value of discount subscribers, just set the reward for the no-discount version to 2 and for the discount version to 1. What matters is the ratio between the numbers, not the absolute value itself (2:1 has the same meaning as 50:25). This also needs to be taken into account in the evaluation, as achieving twice the big conversion rate with the discount variant has the same business value.
Use incremental reward as the customer is moving closer to the final goal.
For example: product view -> add to cart -> purchase
The optimization target is the action that triggers sending the reward to contextual personalization. By default settings, this target directly in the contextual personalization settings is turned on. This works with events tracked on the same page as the weblayer is displayed. If you need to track the reward later or are not on the same page, turn this option off and use the API to track the reward whenever needed. Make sure it is no later than 15 minutes as this is the maximum delay between serving the weblayer and tracking the reward. Keep in mind that the bigger the conversion rate from impression to this target is, the faster it will take the contextual personalization model to learn relevant insights and use them.
The data.contextual_personalization is present only for variants served by the contextual personalization. Therefore, do not try to get reward for variants served by the comparative A/B test, in cases where you use it.
Sending the reward token can be easily done via a simple on_event triggered scenario:
In this example, after every purchase, a verification of whether the user was impressed by the contextual personalization is done. If yes, the reward is tracked.
Updated about 2 months ago