Step 2: Send Your Data - Bloomreach Experience - The Headless Digital Experience Platform Built for Commerce

Step 2: Send Your Data

 

After formatting your data, you must send it to Bloomreach using one of two delivery methods: direct API payload or SFTP file. There are two modes available to update your catalog data: 

After sending your data, you can also check the job status of the delivery:

Endpoints

Environment Endpoint
Staging api-staging.connect.bloomreach.com/dataconnect/api/v1/
Production api.connect.bloomreach.com/dataconnect/api/v1/

Authentication

An API key will be provided for each environment in your account. This access key must accompany all API requests as a request header.

Example API Key merchantname-1066facc-94fb-450e-9f9c-b66d0b155fd2
Example request header 'Authorization: Bearer merchantname-1066facc-94fb-450e-9f9c-b66d0b155fd2'

 

PUT

When you PUT products into a catalog, you replace the catalog’s data entirely. In PUT mode, the only operation allowed is ADD and the only scope allowed is PRODUCT so that data loading can be parallelized and optimized for ingestion speed. We recommend sending your data through a SFTP file when using PUT mode.

You don't need to update the full catalog daily using PUT, and we recommend using PATCH mode if you need to make frequent updates (more than once per day). We recommend using PUT mode if you are updating a large part of your catalog data, but you should limit using PUT to once per day. In PUT mode, a configurable percentage of malformed products is allowed.

Send Data as API payload

To send your catalog data directly via an API payload, you must convert all of the operations into a single JSON Patch, which is an array of catalog records. The max size of a single API payload is 5MB.

Example API payload

[{
  "op": "add",
  "path": "/products/pid-123",
  "value": {
    "attributes": {
      "title": "Example 123",
      "price": 100,
      "description": "Example product description here",
      "url": "http://www.example.com/example-product-url.html",
      "availability": true
    }
  }
},
{
  "op": "add",
  "path": "/products/pid-456",
  "value": {
    "attributes": {
      "title": "Example 456",
      "tags": ["", ""]
    }
  }
},
{
  "op": "add",
  "path": "/products/pid-789",
  "value": {
    "attributes": {
      "title": "Example 789",
      "category": "Seafood",
      "tags": ["", ""]
    }
  }
},
{
  "op": "add",
  "path": "/products/pid-101112/attributes/availability",
  "value": false
}
]

Send Data as a File Through SFTP

To send your catalog data via a file on SFTP, you must convert all of the patch operations into a file of JSONLines. Each line should only contain one patch operation.

Feed files may be sent uncompressed, or be compressed using gzip.

Example API payload for sending files through SFTP

In the request body, provide the paths to your catalog files as relative paths. For example, if your catalog files are in a folder named "content_en", then the request body should look like the following:

[ 
  "content_en/20200410-090909-products1_staging.jsonl", 
  "content_en/20200410-090909-products2_staging.jsonl" 
]

Example SFTP file

{"op": "add", "path": "/products/pid-123", "value": {...}} //First product record
{"op": "add", "path": "/products/pid-456", "value": {...}} //Second product record
{"op": "add", "path": "/products/pid-789", "value": {...}} //Third product record, etc.
…

SFTP Endpoints

Environment Endpoint
Staging sftp-staging.connect.bloomreach.com (54.211.108.247, port 22)
Production sftp.connect.bloomreach.com (3.82.164.133, port 22)

sftp -i ~/path/to/private_key <merchantname>@sftp-staging.connect.bloomreach.com

Provide Bloomreach with public keys that will need access to each environment. More than one can be provided for each environment, and we advise using different public keys for staging and production.

SFTP directory structure
Any catalog data files should be sent via the SFTP location provided in the SFTP Endpoints section. We suggest organizing catalogs into their own folders to help with debugging. For example, you could create a folder called <catalog_name> and then place files related to that catalog in this folder.

File Naming
We suggest naming your catalog data files to include the catalog name, the type of modification (put/full, patch/delta) and a datetime version identifier. This will help with debugging.

For example, if your catalog is named <catalog_name>, you could name your file 20200410-090909-<catalog_name>_full.jsonl

Storage length
Files sent to the Bloomreach SFTP are stored for 30 days. Files that are older than 30 days will be automatically deleted and cleared.

Sample PUT request 

PUT

https://api.connect.bloomreach.com/dataconnect/api/v1/accounts/account ID/catalogs/catalog name/products

 

  • Replace account ID with your Bloomreach provided account ID.
  • The catalog name is exactly the same as your domain key in your search API requests, and points to where this information will be stored on the backend represented with catalog name and language key. 

Headers

Authorization

Bearer API key

 

  • Replace API key with your API key

Content-Type (for Direct API payload)

application/json-patch+json

Content-Type (for SFTP) application/json

Body

Direct API payload

(JSON Patch format)

[{
  "op": "add",
  "path": "/products/pid-123",
  "value": {
    "attributes": {
      "title": "Example 123",
      "price": 100,
      "description": "Example product description here",
      "url": "http://www.example.com/example-product-url.html",
      "availability": true
    }
  }
},
{
  "op": "add",
  "path": "/products/pid-456",
  "value": {
    "attributes": {
      "title": "Example 456",
      "tags": ["", ""]
    }
  }
}]

SFTP File

(JSONLines format,

file paths must be relative)

[
  "{catalog name}/{catalog1.jsonl}",
  "{catalog name}/{catalog2.jsonl}"
]
Response
{"jobId":"713aa624-1d7c-4e4a-ac23-6463e48a5fdd"}

 

PATCH

When you PATCH products into a catalog, you can make partial updates to the catalog’s data. If you need to send REMOVE patch operations, you should use PATCH mode.

Note: You can index multiple PATCH operations with a single Index API call.

In PATCH mode, if there are malformed products, the entire modification will be considered invalid and not applied.

Send Data as API payload

To send your catalog data directly via an API payload, you must convert all of the patch operations into a single JSON Patch, which is an array of catalog records. The max size of a single API payload is 5MB.

If your JSON Patch array payload is larger than 5 MB, this should either be sent in via a file or multiple subsequent PATCH requests split up. If you are splitting multiple requests, you should only perform one build index request after success of all PATCH requests.

When modifying products in a request, we advise querying for the successful state of the job before performing the index update covered in the next section.

Example API payload

[{
  "op": "add",
  "path": "/products/pid-123",
  "value": {
    "attributes": {
      "title": "Example 123",
      "price": 100,
      "description": "Example product description here",
      "url": "http://www.example.com/example-product-url.html",
      "availability": true
    }
  }
},
{
  "op": "add",
  "path": "/products/pid-456",
  "value": {
    "attributes": {
      "title": "Example 456",
      "tags": ["", ""]
    }
  }
},
{
  "op": "add",
  "path": "/products/pid-789",
  "value": {
    "attributes": {
      "title": "Example 789",
      "category": "Seafood",
      "tags": ["", ""]
    }
  }
},
{
  "op": "remove",
  "path": "/products/pid-789"
},
{
  "op": "add",
  "path": "/products/pid-101112/attributes/availability",
  "value": false
}
]

Send Data as a File Through SFTP

To send your catalog data via a file on SFTP, you must convert all of the patch operations into a file of JSONLines. Each line should only contain one patch operation.

Feed files may be sent uncompressed, or be compressed using gzip.

Example API payload for sending files through SFTP

In the request body, provide the paths to your catalog files as relative paths.

[ 
  "{catalog name}/{catalog1.jsonl}", 
  "{catalog name}/{catalog2.jsonl} 
]

Example SFTP file

{"op": "add", "path": "/products/pid-123", "value": {...}} //First product record
{"op": "add", "path": "/products/pid-456", "value": {...}} //Second product record
{"op": "add", "path": "/products/pid-789", "value": {...}} //Third product record, etc.
…

SFTP Endpoints

Environment Endpoint
Staging sftp-staging.connect.bloomreach.com (54.211.108.247, port 22)
Production sftp.connect.bloomreach.com (3.82.164.133, port 22)

sftp -i ~/path/to/private_key <user>@sftp-staging.connect.bloomreach.com

Provide Bloomreach with public keys that will need access to each environment. More than one can be provided for each environment, and we advise using different public keys for staging and production.

SFTP directory structure
Any catalog data files should be sent via the SFTP location provided in the SFTP Endpoints section. We suggest organizing catalogs into their own folders to help with debugging. For example, you could create a folder called <catalog_name> and then place files related to that catalog in this folder.

File Naming
We suggest naming your catalog data files to include the catalog name, the type of modification (put/full, patch/delta) and a datetime version identifier. This will help with debugging.

For example, if your catalog is named <catalog_name>, you could name your file 20200410-090909-<catalog_name>_full.jsonl

Storage length
Files sent to the Bloomreach SFTP are stored for 30 days. Files that are older than 30 days will be automatically deleted and cleared.

Sample PATCH request

Given below is a sample patch operation request: 

PATCH

https://api.connect.bloomreach.com/dataconnect/api/v1/accounts/account ID/catalogs/catalog name/products

 

  • Replace account ID with your Bloomreach provided account ID.
  • The catalog name is exactly the same as your domain key in your search API requests, and points to where this information will be stored on the backend represented with catalog name and language key. 

Headers

Authorization

Bearer API key

 

  • Replace API key with your API key

Content-Type (Direct API payload)

application/json-patch+json

Content-Type (SFTP) application/json

Body

Direct API payload

(JSON Patch format)

[{
  "op": "add",
  "path": "/products/pid-123",
  "value": {
    "attributes": {
      "title": "Example 123",
      "price": 100,
      "description": "Example product description here",
      "url": "http://www.example.com/example-product-url.html",
      "availability": true
    }
  }
},
{
  "op": "add",
  "path": "/products/pid-456",
  "value": {
    "attributes": {
      "title": "Example 456",
      "tags": ["", ""]
    }
  }
}]

SFTP File

(JSONLines format,

file paths must be relative)

[
  "{catalog name}/{catalog1.jsonl}",
  "{catalog name}/{catalog2.jsonl}"
]
Response
{"jobId":"713aa624-1d7c-4e4a-ac23-6463e48a5fdd"}

 

How to Check Job Status

All modifications to a catalog's products will return a Job identifier such as:

{"jobId":"713aa624-1d7c-4e4a-ac23-6463e48a5fdd"}

Use the Job ID to get the status of that modification:

GET

https://api-staging.connect.bloomreach.com/dataconnect/api/v1/jobs/job ID

 

(Replace job ID with the desired job ID)

Header

Authorization

Bearer API key

 

(Replace API key with your API key)

This will return information about the specific ingestion job, including its status.

{
  "account_id": 9999,
  "catalog_name": "homeoasis_en",
  "created_at": "2021-04-28T13:19:02.000303Z",
  "id": "0e4ccde6-5688-47f5-ba9b-b6ffcdb1010b",
  "started_at": "2021-04-28T13:19:05.000302Z",
  "status": "success", //Possible values: creating, queued, running, failed, skipped, killed
  "stopped_at": "2021-04-28T13:20:22.000204Z",
  "type": "index update" //Possible values: ingest delta, ingest full, index update
}

For now, only the "status", "started_at", "stopped_at" properties should be considered stable.

Values for status property:

creating

queued

running

success

failed

skipped

killed

Values in bold are the ones that will typically be encountered.

 

Next Step

After products have been added or modified in a catalog, you must perform a follow up POST request to build and publish the index to make them viewable in the search API. If you want to configure your attributes, you should do so before publishing the index.

Did you find this page helpful?
How could this documentation serve you better?
On this page
    Did you find this page helpful?
    How could this documentation serve you better?