Add Extraction

You can create an extraction either from scratch or by importing a template or a backup .zip file. created with the export feature. In both cases, you must select a source and destination connection. Extractions let you move data consistently, apply transformations, and automate execution through schedules.

Extraction types

Synchronization

Synchronization creates a live, continuously updated copy of your source tables in a destination database, using the same table names. Because the destination tables must be created by DataSync, this type is ideal when you need reliable, repeatable updates for reporting, analytics, or downstream processing.

Use case	Purpose
Build or refresh a data warehouse	Replicate operational tables into a dedicated reporting database.
Replicate on‑premise data to a reporting server	Maintain a separate reporting copy to reduce load and risk on the source system.
Synchronize operational data to the cloud	Copy data to a cloud database so cloud BI tools can access it with better scalability.
Load legacy data with cleanup/transformations	Standardize formats and values during replication.
Add new data sources over time	Keep additional sources synchronized using repeatable schedules.

Consolidation

Consolidation brings data from multiple sources together into a single destination, giving you one unified dataset to work with. This type is especially useful when you need consistent reporting across companies, sites, tenants, or schemas.

Use case	Purpose
Consolidate multiple databases	Merge company or branch databases into one destination for group‑level reporting.
Combine multiple schemas	Load data from multiple schemas into a unified schema to simplify reporting.
Merge recurring Excel files	Consolidate repeated spreadsheets into a single dataset for analysis.

Migration

Migration loads data into existing destination tables by mapping fields from the source to the target structure. Because the destination schema cannot be altered, this type is best suited for system replacements, integrations, or any scenario where the target tables are already defined.

Use case	Purpose
Migrate data between systems	Load legacy data into the new system’s existing tables.
Build interfaces between systems	Regularly move data into a predefined target structure.

Export

Export creates flat‑file outputs (typically CSV) from your source data. This type is ideal when you need to share data with external systems, support file‑based integrations, or prepare data for processes that cannot connect directly to the source.

Use case	Purpose
Generating output files	Export source data into a flat file (typically CSV) so it can be used outside of DataSync.
Migration using flat files (intermediate step)	Generate a flat file (typically CSV) when direct database‑to‑database loading isn’t possible.

Add an extraction

Create a new extraction

1. From the welcome screen or left navigation pane, select Extractions.
2. Click New.
3. Select the type of extraction and click Next.
4. Fill in the required extraction parameters.
5. Configure execution timeout parameters.
6. Select the source connection and destination connection from the dropdown list.
7. If Schema selection is available, choose the schema associated with the source database for both connections. Some connections do not require a schema.
8. Click Create. You are redirected to the Tables page.

Import an extraction

Use the import feature to load a pre‑defined extraction template or restore a backup created with the Export extraction feature. Some templates are provided with DataSync. If you do not see them, contact your partner for access.

1. From the welcome screen or left navigation pane, select Extractions.
2. Select the Import Extraction icon in the upper‑right corner.
3. Click Choose a zip file and select the export .zip file, or drag and drop the into the dialog.
4. Click Next.
5. Fill in the required extraction parameters.
6. Configure execution timeout parameters.
7. Select the source connection and destination connection from the dropdown list.
8. If Schema selection is available, choose the schema associated with the source database for both connections. Some connections do not require a schema.
9. Click Create. You are redirected to the Tables page.

Settings

Extractions settings

Setting	Description
Description	Enter a name for the extraction.
Unique Identifier	(Optional) Enter a value to distinguish identical keys from multiple sources or to merge duplicates. For example, use different values to keep the same invoice number from each source as separate records, or use the same value to merge duplicates. For consolidation, use the same value in the Column Name field for all related extractions.
Parallel Extractions	Set the number of extractions to run simultaneously. Higher values can improve processing speed but require more system resources.
Data Loading Mode	(Migration only) Select how to load data into the destination.
Log Details	Record a detailed log that includes every process step. Leave disabled to record only a summary log.
Max Number of Days Logged	Set how many days of log entries to retain (1–366). The default is 7 days.
Sample Mode	Test the extraction configuration using a limited number of records per table before running a full extraction. By default, 50 rows per table are used.
Primary Keys dos not contain NULL	Remove NULL logic from the WHERE clause when running Process Deleted Records.
Trim spaces	Remove trailing spaces from string fields.

Timeout settings

Setting	Description
Execution Timeout	Set the maximum wait time (in minutes) for each table operation before timing out on the source or destination.
Validate and Build	Set the wait time for validating connections, checking structures, and building the extraction.
Truncate and Load	Set the wait time for clearing (truncating) a destination table and loading new data.
Incremental Load	Set the wait time for updating only rows that have changed since the last run.
Process Deleted Records	Set the wait time for removing destination rows that no longer exist in the source.

Duplicate an extraction

Use this feature to quickly create a similar extraction with minimal changes — for example, when consolidating multiple sources that share the same configuration.

1. In the Extractions page, select an extraction.
2. Select the Duplicate Extraction icon in the upper-right corner.
3. In the Edit Extraction dialog, change the Description and any other required fields.
4. Click Save.

Delete an extraction

1. In the Extractions page, select the extraction.
2. Select the Delete icon in the upper-right corner.
3. Click Confirm.

note

Deleting a table also removes all related indexes and fields. This action cannot be undone.

Extraction Variables

When building a SQL Query or a Post Execution Script, you may need to reference the schema name to access specific tables. Instead of hardcoding these schema names, you can use the system variables ##SourceSchema and ##DestinationSchema. These variables automatically resolve to the Source and Destination schemas defined in the Extraction configuration popup.

Example

select i.* 
from ##SourceSchema.TableX i
left join ##SourceSchema.TableY c 
    on i.FieldX = c.FieldY