Best practices for using custom signals

This guide explains how to select and format custom signals for ranking algorithm training. These best practices help you improve ranking performance and reach your business goals.

Prerequisites

Review Ranking studio and How to use Ranking studio for a feature overview.

Which signals work best

Continuous, numeric, facetable signals

• Use continuous signals with a wide range of values.
• Avoid binary signals (such as new = 1, old = 0) or low-variability signals, since training doesn't work well with these.
• Configure the signals as numeric and facetable in the Catalogs application.

📘

Note

Add the signals to your product catalog feed at least 14 days before algorithm training.

Signals with diverse values

• Use signals with diverse ranges. Avoid signals where more than half of the products share the same value.
• Avoid ranking-style signals (such as product ratings from 0 to 5 or discount flags), since these don't provide enough variability for algorithm training.

Signals that correlate positively with future revenue

When you choose custom signals, assess whether they correlate positively with the algorithm's revenue goal.

Algorithm training optimizes for future revenue. When you add a signal, the algorithm still optimizes revenue and checks whether the signal improves it. If the signal contributes to higher future revenue, it receives significant weight. If not, it receives negligible weight.

These signals led to measurable RPV lifts for some customers:

• Sales forecast.
• In-store or offline sales.
• Total available stock units.

Given their continuous values, these signals have proven effective.

Notes on specific signals

Newness

The current algorithm doesn't work well with signals about newness. If you want to incorporate newness, use continuous values, although this isn't advised. For example, assign values between 0 and 30 days, where 30 represents products added today, 29 represents products added yesterday, and so on.

Margin

To use margin as a custom signal, pass it as a percentage where higher means better margin, so it has continuous values. If your goal is an RPV lift, passing margin as a signal might not help you get there.

FAQs

How many custom signals can I use?

There's no strict limit, but focus on a few high-quality, diverse signals. Avoid overloading the algorithm with too many low-quality signals.

How does the algorithm adapt to changes?

The algorithm uses performance features as input and decays them exponentially to quickly reflect changes in shopper behavior.

Do the signal weights vary over time, or are they static?

The weights are static unless you retrain the algorithm. To retrain, create a new custom algorithm with the same or updated signals. After you add a new signal and start training, the weights of existing signals adjust, which changes product ranking.

How can I use an attribute as a custom signal?

• Send the attribute at the product level, not the variant level.

• Send the attribute with a positive (non-negative) value and configure it in the Catalogs app as numeric and facetable (dynamic or manual).

• The attribute name can contain only alphanumeric characters (a-z A-Z 0-9) or underscores (_), and can't start with a digit.

• Ensure the numeric value represents the document's relevancy in some form, so it shows a clear order of importance for search ranking. For example, economic status with three categories (low, medium, and high), or a product rating from 0 to 5 where 0 is the lowest and 5 is the highest.

What isn't supported: Values, string or numeric, that don't maintain a relative relation or clear order, such as aisle_number, algorithm_number, and categorical values like color or brand.

📘

Note

• If an algorithm currently uses an attribute as a signal, avoid changing that attribute's configuration or deleting the attribute, since this may affect the algorithm.
• If you don't actively send the attribute via feed after adding it for the first time, allow 7 days for the algorithm to train on this data before you use the attribute as an additional signal.

What's the recommendation for setting signal weights?

In general, add new signals to the Ranking studio and let the machine learning model determine the appropriate balance of signal weights.

When is a good time to change signal weights?

Changing signal weights for an algorithm in production can be a consequential decision. Create a new algorithm variant to assess the impact of signal weight changes, then A/B test the new variant to understand the impact on business metrics.