Creating API endpoint for webhooks

Do you want to connect Bloomreach Engagement to your own RESTful system? Due to some Engagement-specific behavior of campaigns and Bloomreach Engagement's webhooks, we have created a guide that explains how to build an API to use with Bloomreach Engagement “Per customer” webhook.

Overview

This article explains:

• What response codes of your API will mean to Bloomreach Engagement
• What performance is required from your API
• What security elements can be used to protect your data in transit

Instructions

Below you will find a short list of all the requirements. They are also explained in more detail in each respective section.

Your API requirements

• Provides HTTPS endpoint with SSL certificate signed by one of Trusted CAs. Review the trusted CA list.
• Handles concurrent requests, this can be set per webhook.
• Responds in less than 1,000 ms.
• HTTP 200 for success or processing failure.
• HTTP 400 for errors or invalid requests to notify Bloomreach Engagement to stop sending more requests.
• HTTP 500 to notify Bloomreach Engagement to retry the same request later.
• Accepts and responds with JSON.
• Authenticates requests with HTTP authorization header (HTTP Basic Auth described in RFC 2617).

Response codes

HTTP 200

In general, your API will respond with HTTP 200 whenever a request was received, parsed and passed initial validation.

{
    "success": true,
    "result": {
        "code": 1,
        "status": "sent",
        "message: "SMS message sent successfully"
}
}

If request processing fails due to input specific to this single request, your API should return HTTP 200 with an error message.

{
    "success": false,
    "errors": [
        { "code": 1004, "description": "SMS message too long. Limit is 160 characters."
    ]
}

HTTP 4xx

Each response HTTP 4xx will be translated as a campaign event with the status "failed".

The exceptions are the error codes 400, 404, 410, 412, 413, and 418, which prevent the webhook from sending any other request for all chunks.

A chunk of customers means a group of customers that entered the scenario at the same time (Now trigger, Regularly triggered scenarios). For event-triggered scenarios, every customer is a new chunk.

Do not respond with 4xx if the request is invalid only for 1 customer due to webhook personalization. For this, you need to use the "per customer" webhook, which sends and processes for one customer only at a time. However, remember that this increases the load on the API endpoint.

As for the batch webhooks, it is impossible to tell from the response to which of the batched customers it belongs. Therefore, it is tracked for all. Even one customer causing an error when received and processed by the API endpoint that returns a 4xx response will add a 'failure' event for all customers in a batch, even if they did not cause that error on the API side.

Typical examples

• 400 Bad request - request generated by Bloomreach Engagement is invalid due to incorrect Webhook definition
• 401 Unauthorized - credentials are not provided or invalid
• 404 Not found

HTTP 500

HTTP 500 responses indicate that your API is unavailable to process the request. Bloomreach Engagement will attempt to retry the request later.

Retry policy

Bloomreach Engagement will retry on HTTP 500 Internal Server Error or higher, 429 Too many requests and also for infrequent connection issues:

• Timeout
• Reject
• Drop

There is a maximum of 3 retries (total of 4 requests attempts). Batch webhooks with batch limit of 10 or above may do up to 7 retries (8 attempts).

The time between each retry increases with some random factor (the average delays for 4 attempts are 7, 45, and 300 seconds respectively with randomized +-10%, which might change in future without announcement).

However, when Bloomreach Engagement encounters too many unrecoverable errors, it starts aborting Webhook requests. This aborting occurs for customers it hasn't processed yet, to protect itself from exhaustion.

Unrecoverable errors

• HTTP 4xx, except 429 Too many requests
• Exhausts number of retries for recoverable errors*
• Frequent connection issues*content type in requests. However
• Rendering repeatedly fails (Jinja personalization)

*The behavior is very specific and the definition is out of the scope of this article.

It's advised that you build your API idempotent, which requires a unique request identifier and rejects the processing of duplicate requests.

Response format

Bloomreach Engagement supports any content type in requests. However, only JSON and plain text format can be parsed as a response. The use of JSON formatted responses is recommended.

Performance requirements

Response time

Bloomreach Engagement expects your API to respond in a timely fashion. The timeouts are set at 10 seconds by default, and we are no longer aborting subsequent requests.

However, we strongly recommend having latencies below 1 second as larger latencies will make the performance of your campaigns worse and risk getting close to the 10-second timeout. It's not not true that we may retry requests with latencies above 1 second.

Concurrency

Bloomreach Engagement scenarios run in parallel by default. If not set differently, It's recommended to scale your API to handle at least 200 concurrent HTTP requests. You can set the desired maximum of parallel requests in each webhook (See more). In the worst case, you may use HTTP 500 or connection reject, with Bloomreach Engagement retry policy in mind, to throttle the number of concurrent requests.

Security

Authentication

HTTP Basic Authentication is implemented natively. See RFC 2617

HTTPS

Your server must expose the API on a secure endpoint to ensure In-Transit encryption. Bloomreach Engagement requires your server's SSL certificate to be signed by Trusted CA. The common name (CN) of the supplied SSL certificate needs to match the API endpoint.

The current list of Trusted CAs can be found here

IP firewall allowlisting

A pool of outgoing IP addresses can be limited by enabling "Use static IPs" in Webhook configuration, where you will find the list of those IPs.

📘

Note

Configuration of static IPs on instances other than app.exponea.com may differ.

Overuse of static IPs in webhook nodes

Be careful when using static IPs in a large number of webhooks. Try to use them only when necessary. If overused, such API requests can create bottlenecks and slow campaign execution on the whole Bloomreach Engagement instance.

As a rule of thumb, a few thousand API requests with static IPs should be OK, but more than a hundred thousand requests could introduce slower execution.

We recommend:

• Instead of IP allowlisting use HTTPS and authorization header as a main security measure
• If possible, call webhook API requests with static IPs in bulk. This should lower the amount of campaign API requests.