Service Quotas Overview

Discovery feed and indexing systems are subject to certain adjustable service level quotas for processing your product catalogs. The quotas allow Bloomreach to maintain a high level of service reliability.

👍
The quotas are primarily applicable to large catalogs i.e. if you have greater than 500,000 documents or 1 GB of data.

Please contact our Support team if you’d like to request a higher quota. Our team can adjust the limits if the request fits within the expected product usage.

Quotas

There are certain quotas that apply only to catalogs using versions 2 and 3 of our indexing system, while there are certain quotas that are applicable only to version 1. You can determine the indexing system version by checking the job details of the most recent index job in Catalog Management. If you are not already using Catalog Management, please contact our Support team to enable the Catalog Management feature.

The table below covers each quota for all the indexer versions, along with the initial default limit and the description of how it functions:

Quota Name	Default Limit	Description
Catalogs on indexer version 2 or 3
Feed record count	500,000	The maximum number of feed records allowed in a single catalog. This is inclusive of product and variant records. It does not take into account any view information such as membership or overrides. In Catalog Management, the current value can be calculated by navigating to the Feed tab and summing the "Product records" and "Variant records" count.
Delta feed patch operation count	10,000	The maximum count of patch operations allowed in a single delta feed update. In Catalog Management, this is found in the Jobs tab as a statistic called patch_operations_count for each Delta feed job.
Document count	500,000	The maximum count of documents allowed in the catalog's index. This is inclusive of: product and variant documents - additional documents needed to handle multi-view override scenarios - invalid documents such as those without url, title, or pricing information and those with availability set to false In Catalog Management, this is reported at the top of each catalog as "Documents". We only count documents in Production for licensing. Note that in the Staging environment, the documents cannot exceed 1.5 times the licensed amount used in Production.
Total documents byte size	1,000 Megabytes	The maximum byte size of the customer supplied data in the index. This is inclusive of: product and variant documents - additional documents needed to handle multi-view override scenarios - invalid documents such as those without url, title, or pricing information and those with availability set to false In Catalog Management, this is reported at the top of each catalog as "Size". Notes: The calculation is similar to counting the bytes of a JSON document string representation of the product and variants and their associated multi-view representations. The size calculation algorithm can be shared with customers if need be and run independently. This calculation only occurs during an index update that reindexes the entire set of data. This is typically after a full feed has been sent and then indexed or the first index update after a configuration change has been made.
Searchable custom attributes byte size	500 Megabytes	The maximum searchable bytes allowed in a catalog's index that come from custom attribute values. If a custom attribute is configured as "Searchable", the value of each attribute will be counted towards the overall byte size. For instance, if the only attribute configured as searchable was named "custom_title" and there were two documents in the index with this attribute, then the searchable byte size would be the sum of each value of "custom_title" across the two documents. In Catalog Management, the custom attributes configured as “Searchable” can be found in the attributes tab. The current value is not presently shown in Catalog Management, but can be found directly in the UI API response of the catalog listing. Note: This calculation only occurs during an index update that reindexes the entire set of data. This is typically after a full feed has been sent and then indexed or the first index update after a configuration change has been made.
Displayable custom attributes byte size	250 Megabytes	The maximum displayable bytes allowed in a catalog's index that come from custom attribute values. If a custom attribute is configured as "Displayable", the value of each attribute will be counted towards the overall byte size. For instance, if the only attribute configured as displayable was named "custom_title" and there were two documents in the index with this attribute, then the displayable byte size would be the sum of each value of "custom_title" across the two documents. In Catalog Management, the custom attributes that are configured as "Displayable" can be found in the attributes tab. The current value is not presently shown in Catalog Management, but can be found directly in the UI API response of the catalog listing. Note: This calculation only occurs during an index update that reindexes the entire set of data. This is typically after a full feed has been sent and then indexed or the first index update after a configuration change has been made.
Facetable custom attributes byte size	1000 Megabytes	The maximum facetable bytes allowed in a catalog's index that come from custom attribute values. If a custom attribute is configured as "Manual” Facetable, the value of each attribute will be counted towards the overall byte size. For instance, if the only attribute configured as facetable was named "custom_title" and there were two documents in the index with that attribute, then the facetable byte size would be the sum of each value of "custom_title" across the two documents. In Catalog Management, the custom attributes that are configured as Facetable can be found in the attributes tab under the Facetable column and marked as “Manual”. The current value is not presently shown in Catalog Management, but can be found directly in the UI API response of the catalog listing. Note: This calculation only occurs during an index update that reindexes the entire set of data. This is typically after a full feed has been sent and then indexed or the first index update after a configuration change has been made.
Dynamic facetable custom attributes byte size	1000 Megabytes	The maximum dynamic facetable bytes allowed in a catalog's index that come from custom attribute values. If a custom attribute is configured as "Dynamic” Facetable, the value of each attribute will be counted towards the overall byte size. For instance, if the only attribute configured as Dynamic Facetable was named "custom_title" and there were two documents in the index with that attribute, then the dynamic facetable byte size would be the sum of each value of "custom_title" across the two documents. In Catalog Management, the custom attributes that are configured as Dynamic Facetable can be found in the attributes tab under the Facetable column and marked as Dynamic. The current value is not presently shown in Catalog Management, but can be found directly in the UI API response of the catalog listing. Note: This calculation only occurs during an index update that reindexes the entire set of data. This is typically after a full feed has been sent and then indexed or the first index update after a configuration change has been made.
Catalogs on indexer version 1
Collection documents byte size	500 Megabytes	The maximum size in bytes of the internal collection that powers the catalog.
Collection document count	500000	The maximum count of documents that powers the catalog. Note: If a catalog uses variants and does not use SKU Select, then this count only references the number of product documents. If a catalog does use SKU Select, this count references both product and variant documents combined.

Error Messaging

If a job exceeds a quota, the job will fail to process and present the user with an error message specifying:

the quota that was exceeded,
the quota value,
the amount by which it was exceeded (if available).

This information provided is illustrated below:

You can track jobs status in the Catalog Management application UI under the jobs tab.

If you are not currently using Catalog Management and use DevStudio's Feed Status, the job listing will show the failure status but does not include the complete error message.

Impact on Existing Customers

These quotas apply to all product catalogs in Discovery. For existing customers, we first determine the current usage. Then quota adjustments are made as follows:

If your current usage is below the default, no adjustments to quotas are made.
If your current usage is greater than the default, adjustments for your specific catalogs have been made, and the new limit is set to 1.25 times the current usage. An example is illustrated below.

As of Sept 19, 2022, if a catalog's current usage of any of the quotas already exceeded the default limit, a quota adjustment was made for the catalog and set at 1.25x the catalog's current usage value. For instance, if your catalog had 600,000 documents on September 19, 2022, a quota adjustment for the catalog was made and set to 750,000 documents (600,000 current documents * 1.25).

If your usage on this catalog were to exceed 750,000 documents, then your indexing job would fail due to hitting the service quota on Document Count, and you would need to contact support to have it increased beyond 750,000.