The Ranking Diagnostics tool is a utility that provides insight into how products in search results are ranked. The tool runs a specific search and returns the search results to you with data that indicates how each piece of product information contributes to the ranking of products in search results. 

Diagnostic information is roughly divided into three parts -

1. The top part of the page gives you query processing information. It includes data like whether the query is autocorrected or relaxed and the synonyms related to the entire collection of search results. It also shows other configured rules that might be affecting the search results.
2. The bottom part of Ranking Diagnostics displays the search results with a ranking diagnostics summary for each product. You can click on a product to view more details.

3. The third part of Ranking Diagnostics is the individual product information, when you click on a product. This page gives specific metrics and scores on the product in question so you can better understand why the product shows up for the given search query and why it is placed at its current position.

Ranking Diagnostics Capabilities

Ranking Diagnostics tool makes debugging your search results intuitive and easier. It provides more information about the search results and makes the overall usage experience smoother and faster. Here's an overview of the major capabilities:

1. Enhanced Visual Elements

The visual elements make your interactions with the tool more engaging and visually pleasing. The colors, icons, and typography provide a cohesive experience.

• You can find tooltips containing explanations and information about various components across the app, which are activated by hovering over them. This will help you quickly understand what certain query processing data means, thereby making your workflow more streamlined.

• When you enter a Category ID/Name as the search query, Bloomreach automatically provides you with typehead suggestions to complete the query.

• The product cards in the search results show labels displaying merchandising operations (Boosted, Buried or Sequentially Locked) applied to that product. This information is valuable if you find inconsistent or unexpected results, which can be due to these operations.

2. Improved Performance and Functionality

Our team has focused on enhancing the overall functionality to make your debugging tasks more efficient and productive. We have added various new functionalities to help you make the most out of the tool and configure it as per your needs, along with performance improvements and faster load times.

• You can now multi-select each available filter to refine your search results and analyze how they affect your results. By narrowing your focus to a particular subset, finding the root cause of any anomalies becomes easier. This targeted approach expedites the debugging process, saving time and effort.
  You can reset all the applied filters through the "Reset" button at once.

• You can easily switch between different accounts in your organization through the site selection dropdown next to the search bar.

3. New, Improved Synonyms applied Section

The "Synonyms applied" section has been overhauled to help you gain deeper insights into how synonyms influence your search, enabling you to fine-tune your configurations and enhance search relevance.
With a focus on delivering enhanced usability and self-understanding, these updates empower you to analyze the synonyms data effectively.

• There can be multiple types of synonyms associated with a search query. These are distinguised based on their origin and how they are formed. Here are the 4 synonym types that you can encounter:
  • User created/modified: Synonyms that you have manually created as custom rules in the dashboard
  • Canonical: Synonym which is a variant of a word to its base form
  • BR generated: Synonyms applied from the pool of AI-generated synonyms by the Bloomreach Engine, which are created based on user activity across your and other Bloomreach customers' sites
  • Derived: Synonyms that get generated based on a combination of other types of synonyms
• Every Derived synonym has a "View details" button next to it that shows the derived synonym's step-wise formation. On the popup containing the details, the underlined words show an association between two synonym rules, and the bolded word is the synonym that finally gets applied.

4. Revamped Modern and Responsive Design

Our new user interface sports a sleek and modern design, making it easier to find the things that are important. It offers a seamless experience across different screen sizes, ensuring you can work with ease.

• Navigating through the product grid has been made more user-friendly by showing paginated results, which can be controlled by selecting the items you want to be displayed per page. The grid has also been made fully responsive to cater to different screen sizes.

• We have collated all additional settings and rules to test your queries under Additional scope definitions. You can now conveniently access these in a centralized location, so you're not left looking and can focus on the direct factors affecting the search results. You can also use these settings to refine your search and test the search results for different audiences.

• The Apply merchandising ranking rules toggle lets you switch between viewing search results with and without all the applied ranking rules. Note that the toggle button does not change the actual status of the ranking rules. This can be controlled more finely by disabling merchandising rules on the API.

  

Step 1: Running a Search

To begin with the debugging process, you should run a search in the Ranking Diagnostics tool. It's located in the Search & Merchandising → Operational tools → Ranking diagnostics section of your Bloomreach Dashboard. Use the Query selection dropdown to run a search query with keywords, a specific product ID, category name or category ID as per your requirement. The diagnostic tool displays your search's results with additional information to help you understand why particular products are showing up in search results and why they are in the sequence displayed.

Step 2: Understanding Query Processing

After you run a search, the top part of the diagnostics report shows the Query Processing section, which highlights factors that affect your search results. Here's an example showing the query, "sandal", and the various features that may have affected that query:

The Query Processing section is divided into 2 main parts: "Synonyms applied" and "Other". Here's an overview of both of them and the information they provide about how the query was processed:

Synonyms applied

This section lists all synonyms that are being applied to the query and displays both the left-hand side and right-hand side of the synonym. The synonyms are distinguished based on their origin as follows:

• User created/modified: Synonyms that you have manually created as custom rules
• Canonical: Synonym which is a variant of a word to its base form
• BR Generated: Synonyms applied from the pool of AI-generated synonyms by the Bloomreach Engine, which are created based on user activity across your and other Bloomreach customers' sites
• Derived: Synonyms that get generated based on a combination of other types of synonyms

📘

Understanding the derived synonyms information

By clicking on View details, you can see how a derived synonym is formed. The underlined words over there show association between two consecutive rules, and the bolded word is the synonym that finally gets applied.

This should make it easier to find the source synonym that could be introducing products into the recall set. 

How does Bloomreach derive synonyms?

Bloomreach continually monitors which terms your customers enter into search boxes and which items they click following the search. When traffic between a search term and a clicked item reaches a threshold, then Bloomreach automatically generates the appropriate synonym rule, adding it to the BR generated Synonyms list on the Synonyms page.

What can I do if I spot a synonym that doesn't belong?

You can reject derived synonyms by removing their association if they don't make sense for the products. If there's a User created or BR generated synonym that doesn't make sense, then you can modify the rule that created it from Synonyms page on the dashboard.

Other

This section provides insights on any other rules/configurations that are associated with that particular query and additional information about the query processing (if applicable).

Query Spell Corrected To

If a query is being autocorrected, this field will display the term that the original query is being autocorrected to. A query is only autocorrected when the original query returns null results.

📘

If the query has not been autocorrected, you won't see this section.

Query Relaxed To

If a query is relaxed, this field will display the value for what the original query was relaxed to (eg: “black” - optional, “hiking” - optional, “boots” - required). A query is relaxed when the original query returns null results and no autocorrect match is found. You can hover over the information icon in the dashboard for more details.

📘

If the query has not been relaxed, you won't see this section.

Step 3: Understanding Product Metrics

After you run a search, the bottom part of the page gives you a brief summary of information about each product in the search results. Here's an example for the query, "office chair":

You can click a product to open a detailed summary of how Bloomreach used information about the product when the search results were ranked. Use this information to analyze why that particular product is part of the search results.

Performance metrics or Scores in Ranking Diagnostics have also been simplified into the following three components:

• Total Score : is the metric based on which products are ranked for a specific query/category. It is a combination of the Performance (User Behavior) and Relevance Scores. The Performance score has a much heavier weight on the Total Score than the Relevance Score.
• Performance : Normalized metric based on user behavior signals such as revenue, conversions, add-to-cart, views etc. generated by a specific product for a specific query/category.
• Relevance : Based on Bloomreach Semantic Engine and Term Match to product feed data (e.g. title, category, brand etc.)

Searchable Feed Data: this table will show you where the user query (or synonyms for the user query) are matching to content in the feed. This section can be used to answer questions like "Why is this product showing up for this query?"

🚧

Please note that SKU-level fields are currently not supported. If there's a match on a SKU-specific field, the value that the query matched on will not be shown in the table.

Diagnose Group Rules

Ranking Diagnostics allows you to preview grouped recall, debug group ranking, and understand the ranking of products within groups.

View Grouped Search Recall

After completing Step 1 and Step 2, check the Group-by checkbox and choose a grouping attribute from the dropdown.

This retrieves grouped recall for the entered Query, Product ID, Category Name, or Category ID. Each group card displays:

1. Group attribute value. For instance, Texture: Soft.
2. Top product of the group and product data like title, price, and product_id.
3. Total score, performance score, and relevance scores of the best-performing product of the group.

📘

Future Improvement

On the Diagnostics page, you may see Synonyms from a previously searched query for a group rule. We will improve this in an upcoming release.

Debug Ranking of Groups and Products Within Groups

How are Groups ranked?

Groups are ranked based on the score of the best-performing product.

Impact of merchandising operations on the ranking of groups

1. Operations applied to Groups

• Group operations take the highest precedence. Suppose there is a group A ( with Product1, Product2 ) and group B (with Product3, Product4 ). Considering the top-performing products, If Product1 has a score of 50 and Product3 has a score of 60, group B is ranked first by default.

However, if you sequentially lock group A in position 1, it takes the highest precedence.

• The same applies to Group boost/bury. Group block will impact the recall as well.
• Attribute-based rules: Attribute include/exclude rules impact the recall and group ranking. Attribute boost/bury rules also impact the group ranking.

2. Operations applied to Products within the Group

• These operations only affect the ranking of products within a group. Boost/Bury/Sequential lock on products within a group will not change the group's ranking. Suppose group A is ranked in position 1, and group B is in position 2.

If you boost Product4 and the new score is 100, there is no impact on the group ranking.

• In the case of a product blocklist, if the first product is blocked from the recall, it will impact the group's ranking.

View Products in a Group

Clicking on a group takes you to the group products page, which displays all the products belonging to that group.

How are Products ranked?

• Products within the group are ranked based on descending total scores.

• The top-performing product of each group is usually placed first unless there's a merchandising rule to change the order of products in the group.

• The group card displays product-level merchandising operations, such as boost, bury, etc.

  

  Inspect individual product ranking

  Click on the individual product in the group to see detailed debugging information.

FAQs

How can I see the impact of other custom-defined rules on the search results?

There can be various account-level customizations and rules directly affecting your search results. Use the Site selection dropdown to select a particular account that you're concerned with. This will only apply the rules defined for that account to your searches, making it easier to point out the exact customizations that are in effect. Any rules affecting your search will be displayed under the Additional scope definitions section.

It is also possible to edit the parameters for these rules in the Ranking Diagnostics tool itself to make more targeted serches and analyze the impact of your customizations. Click on the edit icon next to Additional scope definitions to edit the parameters associated with these rules.

To see the impact that personalization has on a specific user, enter the user's Bloomreach cookie ID in the User ID field inside Additional scope definitions. The user ID is the Bloomreach-assigned value in the user's browser cookie (br_uid_2). The Diagnostics view highlights personalization ranking changes by showing a personalized icon under the product in the results. Use the Audience dropdown to see the impact on a group of users.

What do the diagnostic metrics mean?

Bloomreach derives each returned product's position in search results from a set of signals. Each signal has a normalized score from 0 to 100 that is calculated specifically for that query and result set. A score of 100.0 indicates that the product had the highest score among the result set for that particular metric. When you run a specific search query in the Ranking Diagnostics tool, Bloomreach returns these signals and values.

What can I do with this information?

The Ranking Diagnostic tool helps you understand why a product is showing up in search results. Use the information to better understand the drivers behind a particular result set in Bloomreach, and determine whether any adjustments must be made. The insights gathered from the Ranking Diagnostics panel can help you determine whether a synonym needs to be removed, a particular product attribute in the feed is incorrect, or some products need to be boosted or buried. You can then take action using the other Bloomreach Dashboard tools.