Snowflake Integration ensures the utilization of your data with Snowflake´s standard-based functionality. Your Data Warehouse and BI teams can then use preferred data modeling and query design tools to prepare exports without the risks of any data leaks. The benefit of integration Snowflake is that your data are under the full control of your data teams!

This guide helps you understand how to do both imports and exports using Bloomreach Engagement together with the integration, and it provides you with all of the available options on how to set it up correctly.

📘

Data Freshness Configuration

Bloomreach Engagement allows flexible settings for the frequency of Snowflake imports & exports. For optimal performance, we recommend to set any interval up to every 10 minutes.

Import from Snowflake

This section shows you how to start importing with Snowflake Integration. Imports are near real-time and support scheduled mode.

Step 1: Getting your data ready

First, you need to get your data ready and store them inside CSV files. Creating CSV files can be automated inside Snowflake. Here, you can write a SQL query to select relevant data from your databases and store it in CSV files that are stored on GCS/S3/SFTP. This task is called Unload and it supports bulk exports of your data from a database table into text files.

📘

To check what tasks can be used for unloading, Snowflake provides you with their Overview of Data Unloading and Introduction to Tasks articles.

Step 2: File storage

GCS/S3 setup is needed as well. If you have one that you already use for getting data from other systems, it is recommended to keep using this one. If you do not have GCS/S3, Bloomreach Engagement can provide you with a file storage as a special add-on too.

📘

Snowflake supports splitting the files based on size, and since it is full SQL, can also do incremental delta updates.

Step 3: Execution of imports

Snowflake ensures that only one instance of a task with a schedule is executed at a given time. If a task is still running when the next scheduled execution time occurs, then that scheduled time is skipped. Please make sure that your query execution times are correctly measured and estimated with proper frequency and incremental timestamps. Then, the setup of regular file-based imports from GCS/S3 in Bloomreach Engagement is established.

👍

Tip:

If you want to bulk unload data into Google Cloud Storage, please read this article by Snowflake to get more details on this topic.

Export to Snowflake

Exports are similar to imports, just in reverse. However, they cannot be done in real-time, only in scheduled mode. Follow the steps:

1. Export your data from Bloomreach Engagement using the Exports to GCS option.
2. Keep your data in a GCS-based data lake.
3. If you need to import data into Snowflake database tables from files, use LOAD. Chek out more details in this overview article provided by Snowflake.
4. Trigger automatic loading of files from Engagement by calling the public REST API endpoints. More details can be found in this article.

👍

Tip:

If you want to bulk load data from Google Cloud Storage, please refer to this article by Snowflake to get more details on this topic.