Running and Monitoring a Migration
The Run export section starts the migration and reports on it. It uses every record in the result set you came from, with the mappings and options currently configured.
Export options
| Option | Meaning |
|---|---|
| Transfer File Attachments As Well? | Copies each record's file attachments to the target. Shown only for target systems Cortex can attach files to - FreshService and Halo. |
| Run export as background process | Creates a background task instead of running the export in the dialog, so you can leave the screen or close the browser while it runs. |
| Concurrent records | How many records are submitted to the target at the same time, from 1 (the default) to 10. |
Concurrent records is worth raising when each individual request is slow but the target has spare capacity. Data sources with a rate limit still pace their combined requests, so a higher number does not push a rate-limited target harder - while data sources without a rate limit rely on this number alone to bound concurrent calls. The option is disabled, and the setting has no effect, when the Cortex database is SQLite, because SQLite does not accept concurrent writes.
Running in the dialog
Click Run to export in the foreground:
- Progress is shown as Export progress with a record count ("120 of 480 records") and an estimate of the time remaining.
- While the export runs, Run is disabled. Stop export ends it, and Continue in Background hands the remaining records to a background task without losing the work already done.
- When it finishes, the screen reports "Export finished successfully. N record(s) exported."
Running in the background
With Run export as background process ticked, Run creates a background task and returns immediately. Background migrations appear under Admin → Background Tasks:
- The grid lists each task with its name, status, type (Migrate Data or another type), run context, and a summary.
- A task started from the Migrate Data screen shows a run context of Foreground.
- The Actions column offers icons to pause, resume, and edit the task. Pausing a running migration stops further records from being started; records already in flight finish, and resuming picks up where it left off.
Opening a migration task shows three tabs:
| Tab | Contents |
|---|---|
| Details | The task's settings, Edit mappings… (which opens the same mapping screen), and an optional Cortex security group. Background jobs run their data source queries under that Cortex security group, so it decides whose permissions the migration runs with; if it is left empty - or the group is later deleted - the job uses the background worker's default security group. |
| Record Status | The progress of every record, split across Pending, In Progress, Complete, Failed and Accepted Failures tabs, with Pause and Resume controls. |
| Logs | The task's log entries. |
Because each record's outcome is recorded, an interrupted migration does not start over: the records left Pending are still waiting, and resuming the task processes those and nothing else.
How background jobs share the worker
A single background worker dispatches the queued tasks, and it runs several of them at the same time:
- Up to three jobs run at once by default, so a migration does not have to wait for another background task - a second migration, a ServiceNow mirror, or any other job type - to finish first.
- The cap is set with the
ScheduledJobWorker.MaxConcurrentJobsCortex setting: a value from 1 to 16, where 1 restores one-job-at-a-time behaviour. A blank or zero value uses the worker's default of three. - The cap is always treated as 1 when the Cortex database is SQLite, because SQLite accepts a single writer. Concurrency is therefore only observable on a SQL Server, PostgreSQL or MySQL Cortex database.
- Each job is claimed for processing only just before it starts, so a job that never starts is never left stuck In Progress.
Queuing a background export is immediate. Run export as background process starts ticked, and with it ticked Run queues the task and returns straight away - the dialog no longer loads the individual record IDs first, so even a very large export hands over as soon as you click Run. The task's per-record items are built by the worker when the task starts, in batches, so a freshly queued task can briefly show no records under Record Status. If the source data has changed since the export was queued, the re-evaluated selection can differ, and the task's Logs tab notes the difference.
Records needing attention
The target system can reject individual records - a required field with no value, a choice value it does not accept, a lookup it cannot resolve, a rate limit, a connection problem. Rejected records are collected instead of stopping the run:
- Records needing attention lists each failed record with its source record ID and the Details of the failure.
- Retry failed records tries those records again using the current mapping and export settings. It does not re-run the whole export, so it is safe to fix a mapping, save, and retry.
- Failures that happened while copying related child records are listed per record under Related transfer details - see Related Record Transfers.
- In a background task's Record Status → Failed tab you can retry an individual record, or Accept a failure you do not intend to fix. Accepted failures move to Accepted Failures, are no longer retried, and stop holding the task in a failed state.
A common pattern is to migrate, review the failures, correct the mapping (often a required field left unmapped, or a value translation that is missing a default), and retry only what failed.
Exporting failures to CSV
The Failed and Accepted Failures tabs of a task's Record Status panel each have an Export CSV button. It downloads the whole tab - every failure recorded, not just the page on screen - so a large failure set can be triaged offline or handed to whoever owns the source data. The button appears on those two tabs only, not on Pending, In Progress or Complete.
Each row of the file is one failure, with these columns:
RecordId,Status,Result,TargetParentPublicId,Updated (UTC),FailedAttachments,FailedRelationships,Details
- Result uses the same wording as the badge on the tab, so the file and the screen cannot disagree about why a record failed.
- Attachment and related-item failures are flattened into readable cells, each entry naming the file or child record it concerned together with the message the target returned, and whether a failure was accepted.
- Updated (UTC) is the stored UTC timestamp, so the same download reads identically regardless of the viewer's time zone.
- A row whose stored failure details cannot be read exports the text "Could not read the stored failure details" rather than an empty cell, so a known failure is never hidden by the export.
- The file is named
migration-failures-<task-name>-<tab>-<timestamp>.csv.
See also
- Field Mappings - fixing the mapping behind a rejected record
- Value Translations - providing defaults for unexpected values
- Logging - where Cortex writes its logs if you need to investigate a failure