Best practices for using custom signals

This guide explains how to select and format custom signals for the ranking model training. Following these best practices will help you improve ranking performance and reach your business goals.

Prerequisite knowledge

Review the Ranking Studio guide to get a feature introduction.

What signals work best

Continuous, numeric, factable signals

• Use continuous signals with a wide range of values

• Avoid using binary (such as new = 1, old = 0) or low-variability signals since training doesn't work well with these.

• Configure the signals as numeric and facetable in the Catalogs application.

  

📘

Note

You must add the signals to your product catalog feed at least 14 days before model training.

Signals with diverse values

• Use signals with diverse ranges. Avoid signals where more than half of the products share the same value.
• Avoid ranking-style signals (such as product ratings from 0 to 5 or discount flags), as these don't provide enough variability for model training.

Signals that correlate positively with future revenue

When choosing custom signals, assess if they correlate positively with the model's revenue goal.

Model training optimizes for future revenue. When you add a signal, the model still optimizes revenue and checks whether the signal improves revenue.

If the model computes that the signal you added contributes to a higher future revenue, the signal receives significant weight. If not, it receives negligible weight.

Examples of effective signals that led to measurable RPV lifts for some customers:

• Sales forecast
• In-store or offline sales
• Total available stock units

Given their continuous values, these signals have proven effective.

Notes on specific signals

Newness

The current model doesn't work well with signals about newness. However, if you want to incorporate newness, while not advised, you may use continuous values. For instance, assign values between 0 and 30 days, where 30 represents products added today, 29 for those added yesterday, and so on.

Margin

If you want to use margin as a custom signal, pass it as a percentage where higher means better margin, as it will have continuous values. If the goal is RPV lift, passing margin as a signal might not be helpful to get there.

FAQs

How many custom signals can I use?

There’s no strict limit, but focus on a few high-quality, diverse signals. Avoid overloading the model with too many low-quality signals.

How does the model adapt to changes?

The model uses performance features as input and decays them exponentially to quickly reflect changes in shopper behavior.

Do the signal weights vary over time and fluctuate, or are they static?

The weights are static unless you retrain the model. If you need to retrain, contact your Customer Service Manager.

After adding a new signal and initiating a catalog-specific training, the weights of existing signals adjust, leading to changes in the product ranking.

How can I use an attribute as a custom signal?

• To use an attribute as a custom signal, we expect it to be in a certain format and configuration. Ensure the attribute is sent with a positive (non-negative) value and configured in Catalog Management as ​​Numeric and Facetable (Dynamic or Manual).
• Attribute name must consist only of alphanumeric characters (a-z A-Z 0-9) or underscore (_) and must not start with a digit.
• We also recommend that this numeric value represents the relevancy of the document in some form. This shows a clear order of importance or relevance for search ranking. For example, economic status with three categories (low, medium, and high), product rating with a value range of 0-5 where 0 means lowest and 5 means highest rating, discount, etc.

What isn't supported
Values, string or numeric, that don’t maintain relative relation or a clear order, such as aisle_number, model_number, and categorical values like color/brand.

Notes

• If an algorithm currently uses an attribute as a signal, we recommend not changing the above attribute configuration or deleting the attribute, as this may impact the algorithm.
• If the attribute is not actively sent via feed after adding it for the first time, allow 7 days for our algorithm to train on this data before you use this attribute as an additional signal.

What’s Bloomreach’s recommendation for setting signal weights?

In general, Bloomreach recommends adding new signals to Ranking Studio and letting Bloomreach ML determine the appropriate balance of signal weights.

When is a good time to change signal weights?

Changing signal weights for an algorithm in production can be a consequential decision. We suggest that you create a new algorithm variant to assess the impact of signal weight changes and AB test the new algorithm variant to understand the impact on business metrics.