Home Madrid Now Platform App Engine Import sets key concepts

Import sets key concepts

Learn about Import Sets before importing data to your instance.

The Import Sets table acts as a staging area for records imported from a data source.

Note: Data should not be imported in extremely large chunks. Creating an extremely large import set can cause delays and system outages.

A transform map determines the relationships between fields displaying in an Import Set table and fields in an existing ServiceNow table, such as the Incidents or Users table.

Importing sets will skip records when the data in the instance matches the data being imported.

Note: Import Sets run as user System. Therefore, Import Sets cannot add data to encrypted fields.

Terminology

Table 1. Key terms
Term	Definition
Import Set table	A table that acts as a staging location for records imported from a data source prior to transforming those records. Fields on these tables are generated automatically based on imported data and should not be modified manually.
Data source	A record that defines where to get the data to import. A data source may point to a file, a JDBC-compatible database, or an LDAP organizational unit.
Transformation	The conversion of data from an import set table to another table according to the rules defined in a transform map.
Transform map	A set of field maps that define the relationships between fields in an import set and fields on a table, such as Incident. During transformation, data is copied from the Import Set table to the destination table based on the transform map. A single import set field may be mapped to multiple fields on other tables.
Foreign record insert	A foreign record insert occurs when an import makes a change to a table that is not the target table for that import. This happens when updating a reference field on a table. For example when updating a value for the caller on an incident the import is actually updating the sys_user table.

Supported import formats

You can import data from several different file formats or external data sources.

Table 2. File formats
Format	Limitations
CSV	CSV files must comply with the proposed CSV standard in RFC4180. In particular, this means that double quotes may not appear inside fields. The first row of data in an imported CSV file becomes the header row and defines the columns for that import set.
Excel	Excel files must have the XLS or XLSX extension.
XML	XML files must have a consistent XPath for each data row.
JSON	For JSON arrays, the path for each row must specify the array root element twice, such as `/incidents/incidents`. JSON files should follow RFC-4627. For example, a single comma should separate a value from the following name. Names within an object should be unique. Predicates such as `@element`, `[index]`, or `text()`, as well as Axis such as children, siblings, or ancestors are not supported.

ServiceNow can import data from the following external data sources.

Table 3. External data sources
Data Source	Limitations
JDBC	Some network configurations may require a MID Server.
LDAP	LDAP imports require a valid transform map.

Import set scheduled cleanup

The Import Set Deleter scheduled job cleans up old import set data every day at midnight.

By default, the job will delete all import sets that are older than 7 days and the associated import set table records. If this scheduled job is not active on your instance, activate it to mitigate the growing import set tables that you may have, especially when using recurring imports.

Note: If scheduled cleanup is not currently active on your system, you must ensure that your import set tables are not too big (> 5 million records total across all tables) before enabling this cleanup. If the tables are too big, manually delete the records first before proceeding. Please contact ServiceNow Technical Support to coordinate the deletion of this data.

Previous topic

Next topic

Release version

Import sets key concepts

Learn about Import Sets before importing data to your instance.

The Import Sets table acts as a staging area for records imported from a data source.

Note: Data should not be imported in extremely large chunks. Creating an extremely large import set can cause delays and system outages.

A transform map determines the relationships between fields displaying in an Import Set table and fields in an existing ServiceNow table, such as the Incidents or Users table.

Importing sets will skip records when the data in the instance matches the data being imported.

Note: Import Sets run as user System. Therefore, Import Sets cannot add data to encrypted fields.

Terminology

Table 1. Key terms
Term	Definition
Import Set table	A table that acts as a staging location for records imported from a data source prior to transforming those records. Fields on these tables are generated automatically based on imported data and should not be modified manually.
Data source	A record that defines where to get the data to import. A data source may point to a file, a JDBC-compatible database, or an LDAP organizational unit.
Transformation	The conversion of data from an import set table to another table according to the rules defined in a transform map.
Transform map	A set of field maps that define the relationships between fields in an import set and fields on a table, such as Incident. During transformation, data is copied from the Import Set table to the destination table based on the transform map. A single import set field may be mapped to multiple fields on other tables.
Foreign record insert	A foreign record insert occurs when an import makes a change to a table that is not the target table for that import. This happens when updating a reference field on a table. For example when updating a value for the caller on an incident the import is actually updating the sys_user table.

Supported import formats

You can import data from several different file formats or external data sources.

Table 2. File formats
Format	Limitations
CSV	CSV files must comply with the proposed CSV standard in RFC4180. In particular, this means that double quotes may not appear inside fields. The first row of data in an imported CSV file becomes the header row and defines the columns for that import set.
Excel	Excel files must have the XLS or XLSX extension.
XML	XML files must have a consistent XPath for each data row.
JSON	For JSON arrays, the path for each row must specify the array root element twice, such as `/incidents/incidents`. JSON files should follow RFC-4627. For example, a single comma should separate a value from the following name. Names within an object should be unique. Predicates such as `@element`, `[index]`, or `text()`, as well as Axis such as children, siblings, or ancestors are not supported.

ServiceNow can import data from the following external data sources.

Table 3. External data sources
Data Source	Limitations
JDBC	Some network configurations may require a MID Server.
LDAP	LDAP imports require a valid transform map.

Import set scheduled cleanup

The Import Set Deleter scheduled job cleans up old import set data every day at midnight.

How search works:

Topics are ranked in search results by how closely they match your search terms

Import sets key concepts

Import sets key concepts

Terminology

Supported import formats

Import set scheduled cleanup

On this page

Import sets key concepts

Import sets key concepts

Terminology

Supported import formats

Import set scheduled cleanup

Log in to personalize your search results and subscribe to topics