Import Sets key concepts

Learn about Import Sets before importing data to your instance.

The Import Sets table acts as a staging area for records imported from a data source.

Note: Data should not be imported in extremely large chunks. Creating an extremely large import set can cause delays and system outages.

A transform map determines the relationships between fields displaying in an Import Set table and fields in an existing ServiceNow table, such as the Incidents or Users table.

Importing sets will skip records when the data in the instance matches the data being imported.

Note: Import Sets run as user System. Therefore, Import Sets cannot add data to encrypted fields.

Terminology

Table 1. Key terms
Term Definition
Import Set table A table that acts as a staging location for records imported from a data source prior to transforming those records. Fields on these tables are generated automatically based on imported data and should not be modified manually.
Data source A record that defines where to get the data to import. A data source may point to a file, a JDBC-compatible database, or an LDAP organizational unit.
Transformation The conversion of data from an import set table to another table according to the rules defined in a transform map.
Transform map

A set of field maps that define the relationships between fields in an import set and fields on a table, such as Incident.

During transformation, data is copied from the Import Set table to the destination table based on the transform map.

A single import set field may be mapped to multiple fields on other tables.

Foreign record insert

A foreign record insert occurs when an import makes a change to a table that is not the target table for that import.

This happens when updating a reference field on a table. For example when updating a value for the caller on an incident the import is actually updating the sys_user table.

Supported import formats

You can import data from several different file formats or external data sources.

Table 2. File formats
Format Limitations
CSV CSV files must comply with the proposed CSV standard in RFC4180. In particular, this means that double quotes may not appear inside fields. The first row of data in an imported CSV file becomes the header row and defines the columns for that import set.
Excel Excel files must have the XLS or XLSX extension.
XML XML files must have a consistent XPath for each data row. If you are trying to load an update set XML file, see Load customizations from an XML file.
JSON
  • The XPath for JSON arrays must specify the array root element twice, such as /incidents/incidents.
  • JSON files should follow RFC-4627. For example, a single comma should separate a value from the following name. Names within an object should be unique.
  • Predicates such as @element, [index], or text() as well as Axis such as children, siblings, or ancestors are not supported.

For examples of various JSON formats, see Importing JSON files.

ServiceNow can import data from the following external data sources.

Table 3. External data sources
Data Source Limitations
JDBC Some network configurations may require a MID Server.
LDAP LDAP imports require a valid transform map.

Import set scheduled cleanup

The Import Set Deleter scheduled job cleans up old import set data every day at midnight.

By default, the job will delete all import sets that are older than 7 days and the associated import set table records. If this scheduled job is not active on your instance, activate it to mitigate the growing import set tables that you may have, especially when using recurring imports.
Note: If scheduled cleanup is not currently active on your system, you must ensure that your import set tables are not too big (> 5 million records total across all tables) before enabling this cleanup. If the tables are too big, manually delete the records first before proceeding. Please contact ServiceNow support to coordinate the deletion of this data.
Import Set Deleter scheduled script