Troubleshooting search quality issues

What's in this guide?

Bloomreach uses machine learning algorithms, as well as a huge amount of data, to tune and optimize relevancy on a site automatically. Computers are excellent at rapidly scaling operations, which makes them great optimizers. However, machines lack the capacity to innovate beyond what’s been programmed. This is where people come into the “people + machine” equation to provide innovative inputs for the system to provide better experiences to shoppers.

We recommend you use the scenarios in this guide to quickly fix search terms that are very popular on your site for which results are not relevant. Although uncommon, this sometimes happens for new trends of search terms that the system has not fully learned from user behavior yet. However, if you notice that there is an issue spanning multiple search terms, then we recommend that you report the issue to your Bloomreach contact to explore other potential fixes.

We understand that it is often difficult to predict how a system will react to your instructions. As such, we provide steps to help you determine how to accomplish your goal within Bloomreach. If you have any questions about any of these scenarios, then make sure to reach out to your Bloomreach DXM. 

A product is not shown for any relevant search terms

Scenario

A merchandiser for the dresses category on your website receives a customer complaint. After searching for evening gowns, the customer is unable to find the great dresses they expect your company to carry. You search for the relevant products and can’t seem to find them regardless of the search term you use.

The Bloomreach system keeps a copy of your product catalog in order to understand which products can be included in the result set. We normally update our information each hour based on the latest data provided through your product catalog. If the product is not included in the latest product catalog as available, then we don't show it in search results.

How to test it?

You can quickly check if this product is in our system at all by searching for the product by product ID. When you do this, there are two possible options: 

• The product is returned

  • Indicates that the system is aware of this product and has it marked as a valid product to return. This also means that the product is searchable. 
  • If the product is returned when you search by product ID, but not when you search with another keyword that it should match, then see Scenario: The results for a search term do not include certain relevant products that you carry.

• The product is not returned when searched for by its product ID

  • Indicates that the product is not in the Bloomreach system as a valid, searchable product.

How to fix it?

If the test shows that the product is not in the Bloomreach system, here are a few options:

• Your team that is responsible for the delivery of the Bloomreach data feed needs to check the latest data feed sent to Bloomreach
  • Is the relevant product included in the feed?
    • If not, then work with the data feed team to ensure that the product is added to the feed, assuming it's a valid product.
  • Is the product marked as available or in stock in that feed?
    • Unless there is a special configuration, Bloomreach, by default, only return products that are available to purchase based on the latest data feed.
  • Is the sale price for the product valid?
    • A sale price of $0.00 or null is typically treated as invalid by the Bloomreach system, and the product is not returned.
• When was the last feed sent to Bloomreach in which this product was valid?
  • Depending on your product catalog size, it can take over an hour for the Bloomreach system to reflect the latest data sent
• If the latest feed with the valid product data was sent to Bloomreach over an hour ago, and the product is not being served by Bloomreach for any searches tried, including a search by product ID, then contact Bloomreach Support with details about the product in question and the feed where it was sent for investigation.

The results for a search term do not include certain relevant products that you carry

Scenario

 A customer is looking for trendy dress and doesn't see any results for that search term. You have many dresses in your catalog that are relevant to this term, but they are not being displayed by Bloomreach. They are, however, returned by the system if searched by product ID, product title, or some other related search term.

How to fix it?

There are two potential fixes to make this product part of the search results for a query:

• Add a synonym to match the relevant product. You can expand the result set for the search term in question by adding synonyms. For example, although trendy might not be an attribute you have in your catalog for any of your dresses, you might describe the relevant products as fashion-forward or contemporary. Adding a synonym, such as trendy dress to contemporary dress, ensures that such dresses are retrieved for the trendy dresses search. 
• Ensure appropriate attributes are in the feed as searchable. Determine if there is data you can pass to us in the product feed that includes additional product attributes that we can use to match. For example, you might have a product style that includes attributes such as trendy in it, but Bloomreach currently doesn’t use. If this is the case, then you can work with your Bloomreach contact to add such attributes to the feed and make them searchable within Bloomreach.

If you want to understand more about how synonyms and feed attributes work, read our quick introduction to product retrieval. 

Products that are more relevant are ranked lower in search results

Scenario

There's a product that you want your customer to see. It's showing up in search results, but it's buried below many less relevant products. For example, the customer typed dress, and there are dress shoes in the search results above the dresses that you want higher in the search results.

At a high-level, there are three factors to product ranking: text match, product performance data, and personalization. Of these three, the most common source of issues is the text match component because it is primarily driven by the structure of the product feed and which fields are considered relevant.

How to test it?

You can use the Bloomreach Ranking Diagnostics tool to gather insightful information as to what is driving the ranking of the results you are seeing. 

• Type the query that's producing the odd results in the Ranking Diagnostics tool.
• Look for the product you're interested in and the different signal scores for it: which component of the score is causing the relevant product to show below others? 
  • If it’s a user engagement related score, you might want to consider that this product, although relevant, is not performing very well with your site’s shoppers. 
  • If text match is causing the product to be ranked lower, consider whether there are attributes in the product description that this query should match but isn't.

How to fix it?

Based on your findings, you can consider the following fixes: 

• You can boost specific products, or certain product attributes, through the Bloomreach Dashboard. 
• If text relevance is the issue, then you can consider adding a synonym to capture more relevant terms not being matched by the query today. 
  • For example, let's say that the query is evening dress, the most relevant dresses aren't ranking well due to text match, and the category name for those relevant dresses is formal gowns. Make sure there is a synonym entered from evening dress _to _formal gowns to properly weight the category name in the text relevance formula.

The results for a search term is showing many irrelevant products

Scenario

Consider the situation in which a customer is searching for a product, but many irrelevant products are included in the result set. For example, a customer types sandals, and many dresses are shown in the result set.

Irrelevant products showing up in search results are called “noise” in the results. Noise in search results is typically linked to:

• Attributes or information unrelated to the product provided in the product feed’s searchable fields
• Incorrect synonym applied to the search query

You can read more as to how products are retrieved in Product retrieval.

How to test it?

Using Ranking Diagnostics tool, see why these irrelevant products are matching. You can do this by typing a query and clicking the irrelevant product. Look in the Text Matches section to see how the different keywords in the query matched the product.

• Check product information
  • Check the product description: does it have irrelevant details for the product? For example, maybe for the search, sandals, a dress is coming up because in its long description the dress says “wear this summery dress with your favorite sandals.”
  • When reading through the description, consider the effect of synonyms. For example, the long description might have flip-flops in the long description, and if the user searches for sandals, a synonym from sandals to flip-flops might match that product. 
• Check synonyms
  • Scenario: a user searches for black dress and a green dress is showing up that is not available in black. When looking at the list of synonyms in the Bloomreach Dashboard, you notice that there is a synonym from black to dark. This synonym directs the Bloomreach system to retrieve all products that match black dress as well as dark dress. When looking at the product description for the irrelevant product, you notice that it says dark green as the color. This product was matched because of the black to dark synonym.

How to fix it?

• Synonyms issues
  • If there are bad synonyms causing the match, then consider searching for the bad synonym in the Bloomreach Dashboard’s Synonym page and disabling the synonym in question. This helps reduce the noise. 
  • Consider carefully whether the synonym is a valid synonym. If it is valid, it's often better not to disable it. Disabling a valid synonym can degrade the quality of search results for other queries of product that use this synonym. 
• Product information issues
  • If the source of the noise is product data, you can consider identifying fields in the feed that are currently searchable, but produce noisy or irrelevant data. To disable a searchable field, you need to reach out to the Bloomreach support team at [email protected].

Product retrieval

---

For a product to be returned as part of the results for a particular query, each of the keywords within the query need to match a field within the searchable product information. A query is the user-entered search term after going through the Bloomreach search term interpretation logic, which adds synonyms and removes unnecessary parts. Searchable product information is the data about a product in the Bloomreach system. It's mostly driven by data provided to the Bloomreach platform through a product feed.

Searchable product information

The Bloomreach system built an internal index of your product catalog. In this index, there is an entry for each product in your product catalog with fields that contain all the information about the product. Some of these fields contain information that describes the product and its attributes, which we call searchable__ fields, while others contain information that doesn't specifically describe the product. Consider this sample entry for a hoodie:

In this scenario, we determined that promotional text and flags, although important product information, are fields with content that doesn’t describe the product and so shouldn't be used for retrieval. As a result, the last column shows the promotional text and flags as fields that aren't searchable.

Synonyms, expanded user query

Site search engines need to handle the variability in how users describe a product. While looking for a dress to wear to a formal event, one shopper might search for all of the following with similar intent:

• evening dress
• elegant dresses
• formal gowns
• long night dress

It is the search engine’s job to find the relevant products in the catalog regardless of how they are described.

The most common way to do this is by applying synonyms. A synonym is the correct way to tell the system that elegant dress and evening dress are two ways of looking for the same products.

Let’s see how this works during a search. A shopper goes to a website to find a hoodie, and types the phrase, men dark blue hoodie, in the search box. The search engine has synonyms configured as follows:

• dark blue has the synonym, navy
• men has the synonym, male
• hoodie has the synonym, outerwear

Applying these synonyms to the user-entered query, we can expand the query as follows:

When searching for products in the search results, applying the second synonym allows the system to show navy products in addition to those with dark blue as an attribute. It also matches products with the male attribute in addition to products with men in it. 

Product retrieval process

Putting together the two parts of the scenario in Searchable product information and Synonyms, expanded user query, the full product retrieval process works like this: 

• The user-entered query is men dark blue hoodie
• The expanded query uses the following criteria to match products, which in turn must have the following keywords somewhere in the searchable fields: 
  • men or male
  • dark blue or navy
  • hoodie or outerwear
• Looking at the product information, we see that this criteria is fulfilled as follows: 
  • male matches the searchable Gender field
  • navy matches the searchable Color and Product Name fields
  • hoodie matches the Product Name and Categories fields, and outerwear also matches the Categories field

Since all the criteria for the query to match the product were fulfilled, the product is eligible to be in the search results. However, if the dark blue to navy synonym is disabled, then the product doesn't match because neither dark blue nor dark are in the searchable product information. However, if you disable the hoodie to outerwear synonym, then the product still matches because hoodie already matches the product.