Data Import

A guide to importing data from various sources into BijouCore entities, with a focus on CSV import capabilities.

📄 Overview

Data Import allows you to bring external data into your BijouCore entities. This capability is essential for migrating from legacy systems, integrating with third-party data sources, or bulk loading initial data sets.

The platform provides a flexible, configurable approach to importing data, with support for different file formats, validation rules, and mapping configurations. This ensures that imported data maintains integrity and conforms to your entity requirements.

⚙️ CSV Import

CSV (Comma-Separated Values) is a widely used format for data exchange, and BijouCore provides comprehensive support for importing CSV files into your entities.

Basic Process

The CSV import process follows these general steps:

  1. Create or select an import configuration for your target entity
  2. Upload and preview your CSV file
  3. Map CSV columns to entity fields
  4. Configure import options
  5. Run the import and review results

Import Configurations

Import configurations store your settings for repeated use, including column mappings, import modes, and validation rules.

Property Description
Name Unique identifier for the import configuration
Description Optional descriptive text explaining the purpose of this configuration
Target Entity The entity where data will be imported
Access Permissions Roles that can use this import configuration

CSV Settings

Configure how your CSV file should be parsed:

Setting Description
Delimiter Character that separates values (comma, semicolon, tab, pipe)
Has Header Row Whether the first row contains column names
Skip Rows Number of rows to skip at the beginning
Encoding Character encoding used in the file (UTF-8, ASCII, etc.)
Culture Info Regional settings for parsing dates and numbers
Advanced Options Quote character, line terminators, comment handling, etc.

Import Modes

Control how imported data interacts with existing records:

Mode Description
Create New Only insert new records, do not update existing ones
Update Existing Only update existing records based on key fields, do not create new ones
Create or Update Update existing records if found, otherwise create new ones
Delete Delete records that match the key fields
Synchronize Make the target entity match the source data (create, update, delete as needed)

Column Mappings

Map your CSV columns to entity fields with these options:

Property Description
CSV Column The column name from your CSV file
Entity Field The target field in your entity
Is Key Field Fields used to identify existing records for update/delete operations
Required Whether the field must contain a value

Note: When creating a new entity, you can quickly generate fields from your CSV columns. This is useful for rapid prototyping or when designing entities based on existing data structures.

📊 Import Processing

Validation

During import, data is validated to ensure it meets your entity's requirements:

  • Field data types must match (e.g., dates, numbers, boolean values)
  • Required fields must have values
  • String length limits are enforced
  • Entity validation rules are applied
  • Records with validation errors are logged but not imported

Import Job History

Each import operation is tracked as a job with:

  • Date and time of execution
  • User who initiated the import
  • Number of records processed
  • Success/failure statistics
  • Detailed error logs for troubleshooting

Best Practices

  • Always back up your data before running large imports
  • Test imports with a small sample of your data first
  • Use descriptive configuration names that indicate the data source and purpose
  • Define key fields wisely for update operations
  • Check error logs after import to identify and fix data issues
  • Consider breaking very large datasets into multiple smaller imports