Running and Monitoring a Migration
The Run export section starts the migration and reports on it. It uses every record in the result set you came from, with the mappings and options currently configured.
Export options
| Option | Meaning |
|---|---|
| Transfer File Attachments As Well? | Copies each record's file attachments to the target. Shown only for target systems Cortex can attach files to - FreshService and Halo. |
| Run export as background process | Creates a background task instead of running the export in the dialog, so you can leave the screen or close the browser while it runs. |
| Concurrent records | How many records are submitted to the target at the same time, from 1 (the default) to 10. |
Concurrent records is worth raising when each individual request is slow but the target has spare capacity. Data sources with a rate limit still pace their combined requests, so a higher number does not push a rate-limited target harder - while data sources without a rate limit rely on this number alone to bound concurrent calls. The option is disabled, and the setting has no effect, when the Cortex database is SQLite, because SQLite does not accept concurrent writes.
Running in the dialog
Click Run to export in the foreground:
- Progress is shown as Export progress with a record count ("120 of 480 records") and an estimate of the time remaining.
- While the export runs, Run is disabled. Stop export ends it, and Continue in Background hands the remaining records to a background task without losing the work already done.
- When it finishes, the screen reports "Export finished successfully. N record(s) exported."
Running in the background
With Run export as background process ticked, Run creates a background task and returns immediately. Background migrations appear under Admin → Background Tasks:
- The grid lists each task with its name, status, type (Migrate Data or another type), run context, and a summary.
- A task started from the Migrate Data screen shows a run context of Foreground.
- The Actions column offers icons to pause, resume, and edit the task. Pausing a running migration stops further records from being started; records already in flight finish, and resuming picks up where it left off.
Opening a migration task shows three tabs:
| Tab | Contents |
|---|---|
| Details | The task's settings, Edit mappings… (which opens the same mapping screen), and an optional Cortex security group. Background jobs run their data source queries under that Cortex security group, so it decides whose permissions the migration runs with; if it is left empty - or the group is later deleted - the job uses the background worker's default security group. |
| Record Status | The progress of every record, split across Pending, In Progress, Complete, Failed and Accepted Failures tabs, with Pause and Resume controls. |
| Logs | The task's log entries. |
Because each record's outcome is recorded, an interrupted migration does not start over: the records left Pending are still waiting, and resuming the task processes those and nothing else.
How background jobs share the worker
A single background worker dispatches the queued tasks, and it runs several of them at the same time:
- Up to three jobs run at once by default, so a migration does not have to wait for another background task - a second migration, a ServiceNow mirror, or any other job type - to finish first.
- The cap is set with the
ScheduledJobWorker.MaxConcurrentJobsCortex setting: a value from 1 to 16, where 1 restores one-job-at-a-time behaviour. A blank or zero value uses the worker's default of three. - The cap is always treated as 1 when the Cortex database is SQLite, because SQLite accepts a single writer. Concurrency is therefore only observable on a SQL Server, PostgreSQL or MySQL Cortex database.
- Each job is claimed for processing only just before it starts, so a job that never starts is never left stuck In Progress.
- A job queued while other jobs are already running does not wait for them. The worker re-checks for newly queued jobs about every five seconds and starts them in any free slot under the cap, so a second migration goes In Progress shortly after you queue it rather than after the first one finishes.
Queuing a background export is immediate. Run export as background process starts ticked, and with it ticked Run queues the task and returns straight away - the dialog no longer loads the individual record IDs first, so even a very large export hands over as soon as you click Run. The task's per-record items are built by the worker when the task starts, in batches, so a freshly queued task can briefly show no records under Record Status. If the source data has changed since the export was queued, the re-evaluated selection can differ, and the task's Logs tab notes the difference.
Records needing attention
The target system can reject individual records - a required field with no value, a choice value it does not accept, a lookup it cannot resolve, a rate limit, a connection problem. Rejected records are collected instead of stopping the run:
- Records needing attention lists each failed record with its source record ID and the Details of the failure.
- Retry failed records tries those records again using the current mapping and export settings. It does not re-run the whole export, so it is safe to fix a mapping, save, and retry.
- Failures that happened while copying related child records are listed per record under Related transfer details - see Related Record Transfers.
- In a background task's Record Status → Failed tab you can retry an individual record, or Accept a failure you do not intend to fix. Accepted failures move to Accepted Failures, are no longer retried, and stop holding the task in a failed state.
A common pattern is to migrate, review the failures, correct the mapping (often a required field left unmapped, or a value translation that is missing a default), and retry only what failed.
Forcing a retry of a record's related items
A record can exist on the target while its related items were never sent to it - for example a ticket whose creation reported a failure after the ticket itself had already been created, so the worker never ran its related-item transfers. An ordinary Retry leaves that case alone: it reports the record as failed and tells you to verify the target record manually.
Force retry related items appears above the record grid on a Migrate Data task's Record Status tab and changes what the next Retry does:
- With it ticked, Retry also re-attempts every configured related-item and attachment transfer for a record whose target record already exists, instead of refusing to touch it.
- The instruction is one-shot: it applies to the retry you trigger and is then cleared, so a later retry goes back to the safe behaviour.
- It has no effect on a record that already has recorded transfer failures - those still get the targeted retry that re-attempts only what failed - and none on a record with no target record yet, which is recreated and transferred as normal.
- Related items that did transfer earlier can be sent again, so check the target record afterwards.
Exporting a tab to CSV
Every tab of a task's Record Status panel - Pending, In Progress, Complete, Failed and Accepted Failures - has an Export CSV button on its header row. It downloads the whole tab, not just the page on screen, so a large set of records can be triaged offline or handed to whoever owns the source data.
Each row of the file is one record, with these columns:
RecordId,Status,Result,TargetParentPublicId,Updated (UTC),FailedAttachments,FailedRelationships,Details
- Result uses the same wording as the badge on the tab, so the file and the screen cannot disagree about how a record ended up - Created OK for a completed record, or the specific attachment/related-item failure for one that was created but not fully transferred.
- Attachment and related-item failures are flattened into readable cells, each entry naming the file or child record it concerned together with the message the target returned, and whether a failure was accepted.
- Updated (UTC) is the stored UTC timestamp, so the same download reads identically regardless of the viewer's time zone.
- A row whose stored failure details cannot be read exports the text "Could not read the stored failure details" rather than an empty cell, so a known failure is never hidden by the export.
- The file is named
migration-records-<task-name>-<tab>-<timestamp>.csv, so the tab a file came from is part of the name.
See also
- Field Mappings - fixing the mapping behind a rejected record
- Value Translations - providing defaults for unexpected values
- Logging - where Cortex writes its logs if you need to investigate a failure