Step 2: Send Your Data (Beta) - Bloomreach Experience - Open Source CMS

Step 2: Send Your Data (Beta)

 

In Beta Phase
This feature is currently in the Beta phase. 

After formatting your data, you must send it to Bloomreach using one of two delivery methods: direct API payload or SFTP file. There are two modes available to update your catalog data: PUT (full) and PATCH (delta). 

Endpoints

Environment Endpoint
Staging api-staging.connect.bloomreach.com/dataconnect/api/v1/
Production api.connect.bloomreach.com/dataconnect/api/v1/

Authentication

An API key will be provided for each environment in your account. This access key must accompany all API requests as a request header.

Example API Key merchantname-1066facc-94fb-450e-9f9c-b66d0b155fd2
Example request header 'Authorization: Bearer merchantname-1066facc-94fb-450e-9f9c-b66d0b155fd2'

 

PUT

When you PUT products into a catalog, you replace the catalog’s data entirely. In PUT mode, the only operation allowed is ADD and the only scope allowed is PRODUCT so that data loading can be parallelized and optimized for ingestion speed. We recommend sending your data through a SFTP file when using PUT mode.

In PUT mode, a configurable percentage of malformed products is allowed.

Send Data as API payload

To send your catalog data directly via an API payload, you must convert all of the operations into a single JSON Patch, which is an array of catalog records. The max size of a single API payload is 5MB.

Example API payload

[{
  "op": "add",
  "path": "/products/pid-123",
  "value": {
    "attributes": {
      "title": "Example 123",
      "description": "sample text",
      "tags": ["", ""],
      "language": "en"
    }
  }
},
{
  "op": "add",
  "path": "/products/pid-456",
  "value": {
    "attributes": {
      "title": "Example 456",
      "tags": ["", ""],
      "language": "en"
    }
  }
},
{
  "op": "add",
  "path": "/products/pid-789",
  "value": {
    "attributes": {
      "title": "Example 789",
      "category": "Seafood",
      "tags": ["", ""],
      "language": "en"
    }
  }
},
{
  "op": "add",
  "path": "/products/pid-101112/attributes/availability",
  "value": false
}
]

Send Data as a File Through SFTP

To send your catalog data via a file on SFTP, you must convert all of the patch operations into a file of JSONLines. Each line should only contain one patch operation.

Feed files may be sent uncompressed, or be compressed using gzip.

Example API payload for sending files through SFTP

In the request body, provide the paths to your catalog files as relative paths.

[ 
  "{catalog name}/{catalog1.jsonl}", 
  "{catalog name}/{catalog2.jsonl}" 
]

Example SFTP file

{"op": "add", "path": "/products/pid-123", "value": {...}} //First product record
{"op": "add", "path": "/products/pid-456", "value": {...}} //Second product record
{"op": "add", "path": "/products/pid-789", "value": {...}} //Third product record, etc.
…

SFTP Endpoints

Environment Endpoint
Staging sftp-staging.connect.bloomreach.com (54.211.108.247, port 22)
Production sftp.connect.bloomreach.com (3.82.164.133, port 22)

sftp -i ~/path/to/private_key <user>@sftp-staging.connect.bloomreach.com

Provide Bloomreach with public keys that will need access to each environment. More than one can be provided for each environment, and we advise using different public keys for staging and production.

SFTP directory structure
Any catalog data files should be sent via the SFTP location provided in the SFTP Endpoints section. We suggest organizing catalogs into their own folders to help with debugging. For example, you could create a folder called <catalog_name> and then place files related to that catalog in this folder.

File Naming
We suggest naming your catalog data files to include the catalog name, the type of modification (put/full, patch/delta) and a datetime version identifier. This will help with debugging.

For example, if your catalog is named <catalog_name>, you could name your file 20200410-090909-<catalog_name>_full.jsonl

Sample PUT request 

PUT

https://api-staging.connect.bloomreach.com/dataconnect/api/v1/accounts/{account ID}/catalogs/{catalog name}/products

Headers

Authorization

Bearer {API key}

Content-Type

application/json (SFTP)

application/json-patch+json (Direct API payload)

Body

Direct API payload

[{
  "op": "add",
  "path": "/products/pid-123",
  "value": {
    "attributes": {
      "title": "Example 123",
      "description": "sample text",
      "tags": ["", ""],
      "language": "en"
    }
  }
},
{
  "op": "add",
  "path": "/products/pid-456",
  "value": {
    "attributes": {
      "title": "Example 456",
      "tags": ["", ""],
      "language": "en"
    }
  }
}]

SFTP File

(file paths must be relative)

[
  "{catalog name}/{catalog1.jsonl}",
  "{catalog name}/{catalog2.jsonl}"
]
Response
{"jobId":"713aa624-1d7c-4e4a-ac23-6463e48a5fdd"}

 

PATCH

When you PATCH products into a catalog, you can make partial updates to the catalog’s data. If you need to send REMOVE patch operations, you should use PATCH mode.

In PATCH mode, if there are malformed products, the entire modification will be considered invalid and not applied.

Send Data as API payload

To send your catalog data directly via an API payload, you must convert all of the patch operations into a single JSON Patch, which is an array of catalog records. The max size of a single API payload is 5MB.

If your JSON Patch array payload is larger than 5 MB, this should either be sent in via a file or multiple subsequent PATCH requests split up. If you are splitting multiple requests, you should only perform one build index request after success of all PATCH requests.

When modifying products in a request, we advise querying for the successful state of the job before performing the index update covered in the next section.

Example API payload

[{
  "op": "add",
  "path": "/products/pid-123",
  "value": {
    "attributes": {
      "title": "Example 123",
      "description": "sample text",
      "tags": ["", ""],
      "language": "en"
    }
  }
},
{
  "op": "add",
  "path": "/products/pid-456",
  "value": {
    "attributes": {
      "title": "Example 456",
      "tags": ["", ""],
      "language": "en"
    }
  }
},
{
  "op": "add",
  "path": "/products/pid-789",
  "value": {
    "attributes": {
      "title": "Example 789",
      "category": "Seafood",
      "tags": ["", ""],
      "language": "en"
    }
  }
},
{
  "op": "remove",
  "path": "/products/pid-789"
},
{
  "op": "add",
  "path": "/products/pid-101112/attributes/availability",
  "value": false
}
]

Send Data as a File Through SFTP

To send your catalog data via a file on SFTP, you must convert all of the patch operations into a file of JSONLines. Each line should only contain one patch operation.

Feed files may be sent uncompressed, or be compressed using gzip.

Example API payload for sending files through SFTP

In the request body, provide the paths to your catalog files as relative paths.

[ 
  "{catalog name}/{catalog1.jsonl}", 
  "{catalog name}/{catalog2.jsonl} 
]

Example SFTP file

{"op": "add", "path": "/products/pid-123", "value": {...}} //First product record
{"op": "add", "path": "/products/pid-456", "value": {...}} //Second product record
{"op": "add", "path": "/products/pid-789", "value": {...}} //Third product record, etc.
…

SFTP Endpoints

Environment Endpoint
Staging sftp-staging.connect.bloomreach.com (54.211.108.247, port 22)
Production sftp.connect.bloomreach.com (3.82.164.133, port 22)

sftp -i ~/path/to/private_key <user>@sftp-staging.connect.bloomreach.com

Provide Bloomreach with public keys that will need access to each environment. More than one can be provided for each environment, and we advise using different public keys for staging and production.

SFTP directory structure
Any catalog data files should be sent via the SFTP location provided in the SFTP Endpoints section. We suggest organizing catalogs into their own folders to help with debugging. For example, you could create a folder called <catalog_name> and then place files related to that catalog in this folder.

File Naming
We suggest naming your catalog data files to include the catalog name, the type of modification (put/full, patch/delta) and a datetime version identifier. This will help with debugging.

For example, if your catalog is named <catalog_name>, you could name your file 20200410-090909-<catalog_name>_full.jsonl

Sample Patch request

Given below is a sample patch operation request: 

PATCH

https://api-staging.connect.bloomreach.com/dataconnect/api/v1/accounts/{account ID}/catalogs/{catalog name}/products

Headers

Authorization

Bearer {API key}

Content-Type

application/json (SFTP)

application/json-patch+json (Direct API payload)

Body

Direct API payload

[{
  "op": "add",
  "path": "/products/pid-123",
  "value": {
    "attributes": {
      "title": "Example 123",
      "description": "sample text",
      "tags": ["", ""],
      "language": "en"
    }
  }
},
{
  "op": "add",
  "path": "/products/pid-456",
  "value": {
    "attributes": {
      "title": "Example 456",
      "tags": ["", ""],
      "language": "en"
    }
  }
}]

SFTP File

(file paths must be relative)

[
  "{catalog name}/{catalog1.jsonl}",
  "{catalog name}/{catalog2.jsonl}"
]
Response
{"jobId":"713aa624-1d7c-4e4a-ac23-6463e48a5fdd"}

 

How to check Job Status

All modifications to a catalog's products will return a Job identifier such as:

{"jobId":"713aa624-1d7c-4e4a-ac23-6463e48a5fdd"}

Use the Job ID to get the status of that modification:

GET

https://api-staging.connect.bloomreach.com/dataconnect/api/v1/jobs/{job ID}

Header

Authorization

Bearer {API key}

This will return information about the specific ingestion job, including its status.

{
  "account_id": 9999,
  "attempts": 0,
  "created_at": "1586911454615",
  "started_at": "1586911525950",
  "status": "success",
  "stopped_at": "1586911530597",
  "message": "",
  "error_code": ""
  }
}

For now, only the "status", "started_at", "stopped_at" properties should be considered stable.

Values for status property:

creating

queued

running

success

failed

skipped

killed

Values in bold are the ones that will typically be encountered.

 

Next Step

After products have been added or modified in a catalog, you must perform a follow up POST request to build and publish the index to make them viewable in the search API.

Did you find this page helpful?
How could this documentation serve you better?
On this page
    Did you find this page helpful?
    How could this documentation serve you better?