Data imports

Imports let you bring data from outside the platform directly into your project—whether that's a one-time historical load or an ongoing automated feed.

This guide explains how imports work, what sources are supported, and how to monitor and manage your imports.

Data types

The platform supports four types of importable data. Each has its own import guide:

• Import customers

• Import events

• Import catalogs

• Import vouchers

Data sources

All import types support the same data sources. After selecting a source, always click Preview data to confirm the format is correct. If the preview looks off, check that the right delimiter and encoding are selected.

Data hub catalogs 

For Data hub enabled general and product catalogs, review the supported import types. 

File storage imports

File storage is one of the most secure and convenient ways to automatically transfer data. File storage imports support:

• Filtering files using a regex match. Enclosing slashes / are mandatory to identify a regular expression. for example: /filename[0-9]+/.

• A new file trigger that checks for new files every 5 minutes and imports those not yet processed.

• State memory — the platform tracks all files already uploaded per active import. Deleting an import also clears its remembered files.

Only modify an import's schema—for example, updating a regex pattern or changing the data structure—after all currently scheduled files have been processed. Changing the schema mid-import can cause pending tasks to be cancelled.

Selection rules

When multiple files match your file search, the selection rule determines which files are imported:

• All: All matched files are imported. If already-imported files aren't removed, this causes repeated imports.

• Last modified: Only the most recently modified matching file is imported. Previously imported files are skipped.

• Last alphabetically: Only the last matching file when sorted alphabetically is imported. Previously imported files are skipped.

• Not imported yet: All newly discovered files are imported. The platform tracks which files have already been processed per active import.

📘

Note

The selection rule isn't applied when using the Preview function during import configuration.

Scheduling file imports with the selection rule All and execution rule New file uploaded isn't allowed. Using All or Not imported yet has a hard limit of 200 files—including already-imported files. Clean up your file storage regularly to avoid hitting this limit.

Automatic imports example

You have a DWH that exports data to file storage on a daily basis (for example, purchases). The exported file name will be different every day (regex match) and can be finished at a different time due to the complexity of the export (new file trigger).

Schedule repeated imports

For dynamic sources—File storage, Database, URL, or Bloomreach—you can schedule repeated imports. In the final step of import setup, click Schedule repeated import. Options include:

• Daily: Run multiple times per day.

• Weekly: Run at a set time on selected days.

• Monthly: Run at a set time on selected days each month.

You can also set an end date to automatically stop the repeated import.

Track import progress

After starting an import, monitor it using the progress bar in the import overview. There are 2 stages:

1. Import phase: Data is loaded into the platform.

2. Processing phase: The platform maps and formats the loaded data.

Data is available only after both stages are complete. All imports on an instance share the same processing queue, so large or numerous imports in one project can delay others.

Import run history

The import run listing shows jobs completed within the last 30 days.

Notifications for imports

The platform sends ImportError and ImportRowsDiscarded notifications to all users with the permissionportal-app.imports.update permission. Failed import notifications go to the user who last saved the scheduled import and to everyone with this permission —not necessarily the original import owner.

Roles that include this permission:

• Analyses Editor

• Campaigns Admin

• Campaigns Editor

• Project Admin

• Project Developer

• Technical Support

• Imports Admin

Technical reference

For technical details on deduplication, timestamp handling, ID trimming, specifications for CSV and XML imports, and timeouts, see the Imports technical reference article.

What's next

• Imports best practices

• Imports FAQ