API Guardrails for Added System Reliability

Bloomreach Discovery is a highly scalable service that powers some of the world’s largest E-commerce customers with high uptime. We have helped our customers successfully navigate nearly a decade of peak holiday seasons.

Enterprise stability is at the core of Bloomreach Discovery, which is why we try to remain vigilant with new types of scenarios that can impact system reliability. For example, one of the scenarios that we’ve seen increased occurrences of are bot attacks which can threaten our customer websites and could impact the Bloomreach service. While these situations cannot be completely avoided we have taken measures to protect the system as best as possible.

The Bloomreach Discovery APIs have circuit breakers that are activated when the system detects abnormal thresholds. These thresholds are far higher than what any customer would expect during their regular course of business or during holiday spikes. Below are a list of three guardrails that have been put in place:

Query Type Guardrail Definition
* queries (returns full catalog) Limit set to allow for basic debugging when full catalog needs to be returned
Product Suggest APIs Limit set to allow for all normal usage (protects against automated scrappers & bot attacks)
Search & Category APIs Limits set on a per customer basis. These are continuously monitored and adjusted based on changes to max usage. Limits build in significant buffers to allow for normal/organic increase in usage.

What to expect with normal usage patterns?
Customers should see no interruptions to the Bloomreach service with normal usage patterns. These circuit breaking technologies are designed to allow for normal unimpeded service with organic traffic fluctuations while also protecting against abnormalities.

What to expect when an abnormality triggers a circuit breaker?
When circuit breakers are triggered in our system, all requests within threshold levels will continue to be served as usual through a cache layer. In rare cases, requests served from the cache may return ‘429 Too Many Requests’. This can happen when:

  • The request limits are above the guardrails' thresholds.
  • No cached entry was found corresponding to the request.
  • The API cache is overloaded.

Once all requests are back within the threshold, the 429 responses will automatically stop; normal behavior will continue.

Bloomreach will notify affected customers immediately when circuit breakers are triggered. Our Support team will coordinate with yours to assess the situation. If the usage increase is expected, we can raise the thresholds or keep them in place if the behavior is unexpected.

In cases where you are anticipating a sudden and significant increase in traffic (e.g. from a special promotional event), please inform the Bloomreach Support staff, who can make any adjustments (if needed). For situations where you want to run a load test, please contact your Bloomreach Account team, which can advise you on Performance Environments specifically created for this purpose.

For any questions or further information, contact your Bloomreach account team.