Predictions in Bloomreach Engagement

Predictions offer the following benefits to your business:

  • Insights into future customer behavior - advanced machine learning algorithms accurately predicting customers’ actions
  • Campaign efficiency optimization - predictive insights helping you to achieve your chosen marketing goal by allowing you to target the right customers with the right campaigns
  • Revenue optimization - targeting customer audiences likely to generate the majority of your revenue
  • Omni-channel utilization - option to use predictions across different channels (includes website overlays, onsite personalization, emails)
  • Predictions-enriched customer profiles - individual customer profiles filled with predictions data
  • Easy-to-use predictions builder - ability to build customized prediction models with minimal effort
  • Predefined templates - a ready-to-use model for the most popular predictions
  • Transparent model visualization - easy-to-understand dashboards containing the data on which the predictions models are based

Watch this short introductory video about this feature:

Prediction templates

There the following templates are available in the prediction module:

Purchase predictionPredicts the probability of purchase. It is useful for choosing the audience for the more expensive campaigns. Only those with a high probability of conversion should be targeted through paid channels.
Open email predictionPredicts the probability of an email being opened by a customer. It is useful when you are deciding whether you should or should not send the email to particular customers. This matters because unopened emails decrease their deliverability in the future.
Optimal send timePredicts the best time for sending emails for each individual customer. This maximizes their chances of being opened/clicked.
Churn predictionPredicts the probability of churn. This allows you to take action to prevent it.
In-session predictionPredicts the probability of purchase during the current session. This allows you to nudge the customer to actually make the purchase.
Custom predictionA general template that allows many customizations. It can be used for creating a prediction specifically tailored to your needs. It is aimed at advanced users who are able to take full advantage of the in-depth options and settings that predictions provide. Can predict probability, class, or a value.

For example, you can create the best channel prediction where it chooses the right channels for your customers, so it maximizes your campaigns. Read more about the Best Channel Prediction!

Model quality evaluation

Before a model can be used, you need to evaluate its quality. In order to do so, go to the Results tab and you can check the machine learning metrics.

  • AUC ("Area Under Curve")
    • Area under the ROC (“Receiver operating characteristic”) curve
    • Varies between 0 and 1 (1 is the perfect prediction of the future, 0 is the exact opposite)
    • An uninformative (random) classifier yielding 0.5 for binary classification, or in multiclass classification 1/k, where k is the number of distinct classes we are predicting
  • F1
    • The harmonic mean of Precision and Recall
    • Useful in case of imbalanced classification - typical in our use-cases
    • Range between 0 and 1, 1 is perfect classification, 0.5 is random
  • Matthews correlation coefficient
    • For binary classification
    • MSR
  • Mean squared error - a squared difference of predicted and actual (ground truth) value

Models are always calculated on the training data. Then their accuracy is verified against the testing data. Both datasets are historical, however, the testing one is usually more recent and is not included among the training data even though you dispose of it at the time of the training. Instead, the testing dataset is intentionally not used in the training so that it can be used for accuracy testing. The performance while using the training and testing datasets should be similar (usually, the difference should not be bigger than 10%). Otherwise, it is likely that overfitting had occurred and so the particular model cannot predict the future very well.

The accuracy required from a model will differ according to the context. For example, a model predicting the probability of someone’s ability to pay a mortgage will require a much higher level of accuracy than the prediction about the success of the subscription banner use case.

Visual representation

Bloomreach Engagement predictions return the model of customers' behavior in the form of a decision tree. The tree is a set of rules against which your customers will be compared and which will filter out customers who do not fit them. The tree allows you to see how many customers match a particular rule (be it a certain event, value, or segment) and the statistical probabilities that a customer out of this cohort will finish the chosen goal (for example, purchase).

Reading a predictions node

At the top left corner of each decision node, you can read the rule that it represents. In the example screenshot, the rule includes only customers who purchase at least six items in the last 29 days. Below the rule, you can see the number of customers who fit this rule, which is 323 (0,2% of all the tracked customers). To the right, there is the number and graph portraying how many customers out of these 323 finish the chosen goal. In the example, it is 60,5%.

Reading a predictions tree

You can read a decision tree as a set of rules that separate the target group from the rest of your customers in a given cohort.

Example of a decision tree (predicting that customer will ask for short term loan).

The screenshot portraits an example of a decision tree predicting that a customer will ask for a loan. It consists of 5 splits:

  1. # event loan calculator - if the count is 2 or more then the customer has a probability of 63.09% to take the action. Otherwise, the probability is 28.49% but the customer continues through the tree to the second split.
  2. Last product view - If the customer viewed a card as the last product then their probability for action is 12.11%. Otherwise, the probability is 32,14% and the customer continues to the 3rd split.
  3. # product_view - Based on the # product view the customer continues to 4th or 5th split.
  4. Last first_session device - If the device is one of (BlackBerry, iPad, Other) then the probability is 26.53% otherwise it is 24.64%
  5. Last product_view browser - If the browser from the last product_view is in (Chrome, Firefox, <n/a>) then the probability is 34.58. Otherwise it is 51.5%

There are 6 last nodes in the tree. These are the basis for the final probabilities calculated for different customers according to their customer journeys.

Creating model vs Serving model

To understand how the probabilities are updated, we need to distinguish between creating and serving the model.

Creating a model is the process of creating the decision tree based on historical data explained above. The model is not recreated automatically and you can be using a month's old model if you do not decide to recreate it.

Serving a model is a process of retrieving a predicted value for a particular customer. This data point is always updated to the most current version when the system asks for it, for example, when you look at the value in your customers’ profile or when you use the customer with a prediction value in a campaign. This means that each time you get data points for a particular customer, the newest version of their customer profile goes through the model which had been created some time ago.

Note: model creation uses all data tracked in Bloomreach Engagement regardless of expiration while data serving only uses data visible in the app and so the expiration needs to be considered.

Using predictions

All your created predictions can be found in Analyses > Predictions.

You can also see the predictions’ outputs for each individual customer directly in their customer profile in the Predictions sections.

As you can see in the screenshot above, the name of the prediction attribute is identical to the name of the prediction on which it is based. This attribute can be used in the same way as any other customer attribute across Bloomreach Engagement's different modules - including filters, reports, segmentations, scenarios, etc. For example, in the screenshot below, you can see a segmentation based on the probability of purchase. Afterward, you can use the segmentation in a campaign where you approach only customers with middle to a very high probability of purchase.