Thank you for your feedback.
Form temporarily unavailable. Please try again or contact to submit your comments.
  • London
  • Kingston
  • Jakarta
  • Istanbul
  • Helsinki
  • Geneva
  • Store

Import set performance

Import set performance

The algorithm transforms import sets from their staging table into their final destination.

Importing data via an import set requires two phases:
  1. The data is loaded from a data source into a staging table
  2. The data is transformed from the staging table into a target table

The transform algorithm operates in "blocks" of 100 records at a time as opposed to the previous algorithm which transformed a single record at a time. The newer approach allows the application server to pre-fetch a variety of information relevant to each block of records, reducing the number of unique interactions with the database and improving throughput.

Who should expect to see a performance improvement?

Any customer using import sets should expect to see a performance improvement in large transformations.

What kinds of transformations benefit the most from these changes?

Transformations with a large number of reference or choice type columns see the largest improvement.

What kind of transformations benefit the least?

Transformations that with complex or unkeyed coalesce conditions see a proportionately smaller benefit.

Updates to site content will be made starting around 4am on January 17th (Pacific Time) and lasting approximately 6 hours.  This site may be intermittently unavailable.