Schedule imports to make it possible to specify that certain import operations occur at a regular interval.

Before you begin

Role required: import_scheduler, import_admin, or admin
Warning:

Giving the user the import_scheduler role is equivalent to giving the user the admin role, because the import_scheduler has the ability to execute scripts with administrator level privileges.

Procedure

  1. Navigate to All > System Import Sets > Administration > Scheduled Imports.
  2. Click New.
  3. On the form, fill in the fields.
    Table 1. Scheduled Data Import form
    FieldDescription
    Name Name for the scheduled data import.
    Data source Data source record that defines the data to import.

    If you select a remote data source, it refreshes using the remote connection before the import begins.

    The transform map or robust transformer used for the data import is connected to the selected data source. You can view the transform map or robust transformer by opening the data source record and viewing the Transforms and Robust Transformer tabs.
    Run as Name of a user whose credentials the import job uses. If you do not specify a value, the import job runs as the guest user.
    Active Option to activate the scheduled import.
    Use connection Option to specify the connection alias for the scheduled import. This option is available for Data Sources configured to use a connection alias.
    Connection

    Connection to use for the scheduled import. Select the default parent alias or any of its children's aliases. This field is available when Use connection is checked.

    The sys_id for the selected connection alias is stored in the Glide session. It can be accessed in the script for the custom data source or in any transform script using the code gs.getSession().getClientData('connection_credential_alias_override').
    Run Frequency at which you want to run the import.
    Day Day on which to run the import. Visible only if you select Weekly or Monthly from the Run list.
    Time Time that you want the import to begin. Do not schedule multiple imports at the same time. Running multiple imports at the same time might negatively impact performance or cause an instance outage.
    Repeat interval Period that must elapse before the import repeats. Visible only if you set Run to Periodically.
    Starting Date on which the import should begin (selected using the select date and time icon (Select date and time.). Visible only if you set Run to Periodically.
    Parent The parent import set to which this import set belongs, and after which this import should run. Visible only if you set Run to After Parent Runs.
    Conditional Option to specify a script that defines the conditions that must evaluate to true before the import runs.
    Condition Script that evaluates whether an import should run. Visible only if you select Conditional.
    Concurrent Import Option to run a concurrent import. A concurrent import splits the data into multiple import sets and runs the import sets concurrently to reduce processing time for large data sets.
    Note: A concurrent import does not transform data in a predictable order. If order matters, do not run a concurrent import.

    For more information, see Concurrent imports.
    Partition Method Method used to allocate each data set record to an import set:
    • Round Robin: The system sets partitions automatically.

      Assume that there are 20 records and 4 import sets. The system allocates record 1 to import set 1, record 2 to import set 2, record 3 to import set 3, and record 4 to import set 4. The system then allocates record 5 to import set 1, record 6 to import set 2, record 7 to import set 3, and record 8 to import set 4. This pattern continues until all records are assigned to import sets.

      When the allocation finishes, import set 1 contains records 1, 5, 9, 13, 17. Import set 2 contains records 2, 6, 10, 14, 18. Import set 3 contains records 3, 7, 11, 15, and 19. Import set 4 contains records 4, 8, 12, 16, and 20.

    • Custom Script: Sets the partitions with a Partition Script that you provide.
    • Custom size: Sets the partition size to the Partition Size specified.

      After the system processes the first partition, the data from that partition can be transformed, even though the other partitions have not finished processing.
    Partition Script Script used to return a partition key. The system ensures that data set rows with the same partition key go to the same import set.

    Visible only if you set Partition Method to Custom Script.
    Partition Size The number of records in each partition.

    Visible only if you select a Partition Method of Custom size.
    Execute pre-import script Option to specify a script to run before the import happens.
    Prescript Script to run before the data imports. Visible only if you select Execute pre-import script.
    Execute post-import script Option to run a script after the import happens.
    Post script Script to run after the import happens. Visible only if you select Execute post-import script.
    Note: For concurrent imports, the script runs after the last Import Set Transformer job completes.
  4. Click Submit.
  5. To stop an import that is taking too much time, navigate to System Import Sets > Advanced > Progress.
    1. Open the Progress Worker that you must cancel.
    2. Click the Cancel job related link.
  6. To display a list of executions records, click Executions.
  7. From the Executions list, select the execution record that you want to display.

    The execution record shows:

    • The execution number.
    • The schedule import name.
    • The import set name.
    • The execution context number.
  8. To display details about the executions, select the executions contexts record that you want to display.