Custom (Load by Script) type data source
-
- UpdatedJan 30, 2025
- 3 minutes to read
- Yokohama
- Data Management
Use a custom script to store any type of incoming data in the import set table.
When you select a Type of Custom (Load by Script) for your data source, the Data Loader field appears. The Data Loader field is a JavaScript field where you can enter a script to fetch data and insert it into the import set table using the import_set_table input parameter. For example, you can enter a script to get data from a REST API or a file attachment.
Data Loader script
import_set_table
: The import set table referred to in the Data Source record. The data is inserted into this table.data_source
: The data source referred to in the Data Source record.import_log
: The log that records information about data import activity.last_success_import_time
: The last time this data source was run successfully.partition_info
: The partitioning information for the data. Used for parallel loading.
addColumn(label, maxLength)
: Adds a string-type column to the import set table.addJSONColumn(label, maxLength)
: Adds a JSON-type column to the import set table.addXMLColumn(label, maxLength)
: Adds an XML-type column to the import set table.insert(rowData)
: Inserts a map (key = column name, value = column value) in the import set table.getMaximumRows()
: Returns 20 when you select Test load 20 records. In every other case, returns -1.
Data Loader example
Parallel loading script
parallel_job_loader
: The Parallel Jobs [sys_parallel_job] table.data_source
: The data source referred to in the Data Source record.import_log
: The log that records information about data import activity.
The parallel_job_loader parameter has the add(partitionInfo)
method, which adds the partition information to the Parallel Jobs [sys_parallel_job] table.
Parallel loading example
Parallel loading properties and status
If a job gets stuck in a Running state for more than 600 minutes, it's automatically put back into a Pending state. You can modify the amount of time a parallel job is allowed to be stuck in a running state with the com.glide.system_parallel_job_inactive_time_minutes property.
If a parallel job can't complete after two tries, the job is set to Error. You can modify the number of tries before having an error state with the com.glide.system_parallel_job_max_retry_count property. The com.glide.system_parallel_job_auto_retry property can be set to false to disable auto retry.
You can check the status of your current parallel import tasks by navigating to
, selecting your job, and viewing the Parallel Jobs tab.Script auto-complete
As you type the script, the script auto-complete displays the available options, as shown in this example.