Contents Now Platform App Engine Previous Topic Next Topic Import sets key concepts Subscribe Log in to subscribe to topics and get notified when content changes. ... SAVE AS PDF Selected Topic Topic & Subtopics All Topics in Contents Share Import sets key concepts Learn about Import Sets before importing data to your instance. The Import Sets table acts as a staging area for records imported from a data source. Note: Data should not be imported in extremely large chunks. Creating an extremely large import set can cause delays and system outages. A transform map determines the relationships between fields displaying in an Import Set table and fields in an existing ServiceNow table, such as the Incidents or Users table. Importing sets will skip records when the data in the instance matches the data being imported. Note: Import Sets run as user System. Therefore, Import Sets cannot add data to encrypted fields. Terminology Table 1. Key terms Term Definition Import Set table A table that acts as a staging location for records imported from a data source prior to transforming those records. Fields on these tables are generated automatically based on imported data and should not be modified manually. Data source A record that defines where to get the data to import. A data source may point to a file, a JDBC-compatible database, or an LDAP organizational unit. Transformation The conversion of data from an import set table to another table according to the rules defined in a transform map. Transform map A set of field maps that define the relationships between fields in an import set and fields on a table, such as Incident. During transformation, data is copied from the Import Set table to the destination table based on the transform map. A single import set field may be mapped to multiple fields on other tables. Foreign record insert A foreign record insert occurs when an import makes a change to a table that is not the target table for that import. This happens when updating a reference field on a table. For example when updating a value for the caller on an incident the import is actually updating the sys_user table. Supported import formats You can import data from several different file formats or external data sources. Table 2. File formats Format Limitations CSV CSV files must comply with the proposed CSV standard in RFC4180. In particular, this means that double quotes may not appear inside fields. The first row of data in an imported CSV file becomes the header row and defines the columns for that import set. Excel Excel files must have the XLS or XLSX extension. XML XML files must have a consistent XPath for each data row. JSON For JSON arrays, the path for each row must specify the array root element twice, such as /incidents/incidents. JSON files should follow RFC-4627. For example, a single comma should separate a value from the following name. Names within an object should be unique. Predicates such as @element, [index], or text(), as well as Axis such as children, siblings, or ancestors are not supported. ServiceNow can import data from the following external data sources. Table 3. External data sources Data Source Limitations JDBC Some network configurations may require a MID Server. LDAP LDAP imports require a valid transform map. Import set scheduled cleanup The Import Set Deleter scheduled job cleans up old import set data every day at midnight. By default, the job will delete all import sets that are older than 7 days and the associated import set table records. If this scheduled job is not active on your instance, activate it to mitigate the growing import set tables that you may have, especially when using recurring imports.Note: If scheduled cleanup is not currently active on your system, you must ensure that your import set tables are not too big (> 5 million records total across all tables) before enabling this cleanup. If the tables are too big, manually delete the records first before proceeding. Please contact ServiceNow Technical Support to coordinate the deletion of this data. On this page Send Feedback Previous Topic Next Topic
Import sets key concepts Learn about Import Sets before importing data to your instance. The Import Sets table acts as a staging area for records imported from a data source. Note: Data should not be imported in extremely large chunks. Creating an extremely large import set can cause delays and system outages. A transform map determines the relationships between fields displaying in an Import Set table and fields in an existing ServiceNow table, such as the Incidents or Users table. Importing sets will skip records when the data in the instance matches the data being imported. Note: Import Sets run as user System. Therefore, Import Sets cannot add data to encrypted fields. Terminology Table 1. Key terms Term Definition Import Set table A table that acts as a staging location for records imported from a data source prior to transforming those records. Fields on these tables are generated automatically based on imported data and should not be modified manually. Data source A record that defines where to get the data to import. A data source may point to a file, a JDBC-compatible database, or an LDAP organizational unit. Transformation The conversion of data from an import set table to another table according to the rules defined in a transform map. Transform map A set of field maps that define the relationships between fields in an import set and fields on a table, such as Incident. During transformation, data is copied from the Import Set table to the destination table based on the transform map. A single import set field may be mapped to multiple fields on other tables. Foreign record insert A foreign record insert occurs when an import makes a change to a table that is not the target table for that import. This happens when updating a reference field on a table. For example when updating a value for the caller on an incident the import is actually updating the sys_user table. Supported import formats You can import data from several different file formats or external data sources. Table 2. File formats Format Limitations CSV CSV files must comply with the proposed CSV standard in RFC4180. In particular, this means that double quotes may not appear inside fields. The first row of data in an imported CSV file becomes the header row and defines the columns for that import set. Excel Excel files must have the XLS or XLSX extension. XML XML files must have a consistent XPath for each data row. JSON For JSON arrays, the path for each row must specify the array root element twice, such as /incidents/incidents. JSON files should follow RFC-4627. For example, a single comma should separate a value from the following name. Names within an object should be unique. Predicates such as @element, [index], or text(), as well as Axis such as children, siblings, or ancestors are not supported. ServiceNow can import data from the following external data sources. Table 3. External data sources Data Source Limitations JDBC Some network configurations may require a MID Server. LDAP LDAP imports require a valid transform map. Import set scheduled cleanup The Import Set Deleter scheduled job cleans up old import set data every day at midnight. By default, the job will delete all import sets that are older than 7 days and the associated import set table records. If this scheduled job is not active on your instance, activate it to mitigate the growing import set tables that you may have, especially when using recurring imports.Note: If scheduled cleanup is not currently active on your system, you must ensure that your import set tables are not too big (> 5 million records total across all tables) before enabling this cleanup. If the tables are too big, manually delete the records first before proceeding. Please contact ServiceNow Technical Support to coordinate the deletion of this data.
Import sets key concepts Learn about Import Sets before importing data to your instance. The Import Sets table acts as a staging area for records imported from a data source. Note: Data should not be imported in extremely large chunks. Creating an extremely large import set can cause delays and system outages. A transform map determines the relationships between fields displaying in an Import Set table and fields in an existing ServiceNow table, such as the Incidents or Users table. Importing sets will skip records when the data in the instance matches the data being imported. Note: Import Sets run as user System. Therefore, Import Sets cannot add data to encrypted fields. Terminology Table 1. Key terms Term Definition Import Set table A table that acts as a staging location for records imported from a data source prior to transforming those records. Fields on these tables are generated automatically based on imported data and should not be modified manually. Data source A record that defines where to get the data to import. A data source may point to a file, a JDBC-compatible database, or an LDAP organizational unit. Transformation The conversion of data from an import set table to another table according to the rules defined in a transform map. Transform map A set of field maps that define the relationships between fields in an import set and fields on a table, such as Incident. During transformation, data is copied from the Import Set table to the destination table based on the transform map. A single import set field may be mapped to multiple fields on other tables. Foreign record insert A foreign record insert occurs when an import makes a change to a table that is not the target table for that import. This happens when updating a reference field on a table. For example when updating a value for the caller on an incident the import is actually updating the sys_user table. Supported import formats You can import data from several different file formats or external data sources. Table 2. File formats Format Limitations CSV CSV files must comply with the proposed CSV standard in RFC4180. In particular, this means that double quotes may not appear inside fields. The first row of data in an imported CSV file becomes the header row and defines the columns for that import set. Excel Excel files must have the XLS or XLSX extension. XML XML files must have a consistent XPath for each data row. JSON For JSON arrays, the path for each row must specify the array root element twice, such as /incidents/incidents. JSON files should follow RFC-4627. For example, a single comma should separate a value from the following name. Names within an object should be unique. Predicates such as @element, [index], or text(), as well as Axis such as children, siblings, or ancestors are not supported. ServiceNow can import data from the following external data sources. Table 3. External data sources Data Source Limitations JDBC Some network configurations may require a MID Server. LDAP LDAP imports require a valid transform map. Import set scheduled cleanup The Import Set Deleter scheduled job cleans up old import set data every day at midnight. By default, the job will delete all import sets that are older than 7 days and the associated import set table records. If this scheduled job is not active on your instance, activate it to mitigate the growing import set tables that you may have, especially when using recurring imports.Note: If scheduled cleanup is not currently active on your system, you must ensure that your import set tables are not too big (> 5 million records total across all tables) before enabling this cleanup. If the tables are too big, manually delete the records first before proceeding. Please contact ServiceNow Technical Support to coordinate the deletion of this data.
