Product documentation Docs
    • English
    • Deutsch
    • 日本語
    • 한국어
    • Français
  • More Sites
    • Now Community
    • Developer Site
    • Knowledge Base
    • Product Information
    • ServiceNow.com
    • Training
    • Customer Success Center
    • ServiceNow Support Videos
  • Log in

Product documentation

  • Home
How search works:
  • Punctuation and capital letters are ignored
  • Special characters like underscores (_) are removed
  • Known synonyms are applied
  • The most relevant topics (based on weighting and matching to search terms) are listed first in search results
Topics are ranked in search results by how closely they match your search terms
  • A match on the entire phrase you typed
  • A match on part of the phrase you typed
  • A match on ALL of the terms in the phrase you typed
  • A match on ANY of the terms in the phrase you typed

Note: Matches in titles are always highly ranked.

  • Release version
    Table of Contents
    • Now Platform capabilities
Table of Contents
Choose your release version
    Home Orlando Now Platform Capabilities Now Platform capabilities IntegrationHub Data Stream actions and pagination

    Data Stream actions and pagination

    • Save as PDF Selected topic Topic & subtopics All topics in contents
    • Unsubscribe Log in to subscribe to topics and get notified when content changes.
    • Share this page

    Data Stream actions and pagination

    Send REST or SOAP requests from Flow Designer to APIs that return a stream of response data larger than 10 MB, or that return paginated results. Parse stream data into a series of complex object outputs and use the data pills in other actions in a flow.

    For example, create a Data Stream action to import a large quantity of employee data from a third-party HR site. The Data Stream action sends a REST request to the third-party site and processes the response to populate records in the User [sys_user] table.

    Benefits

    Data Stream actions offer these benefits.

    • Parse and format a stream of response data larger than 10 MB.
    • Automatically send multiple requests to APIs that paginate results.
    • Enable flow designers to process large requests without complex coding or configuration.
    • Enable flow designers to process each object within a data stream using For each flow logic. For example, you might create a Data Stream action that imports document data from a third-party site. When you add the action to a flow, Flow Designer automatically adds the action to a For each flow logic block, enabling flow designers to easily create a record in ServiceNow for each object in the data stream. See Use a Data Stream action in a flow.
    • Enable flow designers to reuse Data Stream actions in multiple flows, using the same source of data in multiple ways.

    Running a Data Stream action

    There are two ways to run a Data Stream action.

    From a flow
    You can process each object within a data stream using For each flow logic. For example, you might create a Data Stream action that imports document data from a third-party site. When you add the action to a flow, Flow Designer automatically adds the action to a For each flow logic block, enabling flow designers to easily create a record in ServiceNow for each object in the data stream. See Use a Data Stream action in a flow.
    From a script
    You can start a Data Stream using the executeDataStreamAction() method in the FlowAPI class. For more information, see FlowAPI.

    Action outline

    Data Stream actions follow a set structure. Follow prompts to add and remove steps from the action outline. You cannot manually add steps to a Data Stream action.

    Animation showing the Datastream Outline and selected options
    Note: Clearing an option in a configuration page removes the step from the Data Stream outline and deletes all data associated with the step.

    Action Preprocessing

    Use the Action Preprocessing category to run a preprocessing script before the action sends the initial API request. For example, validate action inputs or set default values. Preprocessing executes once per action, before the first API request.

    Selecting this option adds a script step to the Data Stream action. For more information, see Script step.

    This is an optional Data Stream action component that runs on either the instance or a MID Server.

    Request

    Use the Request category to configure how the action sends API requests. The Request section executes once per page of results. Request components provide these configuration options.

    Pagination Setup step

    Request results in batches. Once one page of data is processed, the Data Stream action runs the request section again to return the next set of results. Use the pagination setup step to set up pagination options required by the API. Configure the Pagination Setup step manually, or select a pre-built template to apply common configurations. For example, apply the Limit / Offset template to specify the number of items you want returned per page (limit), and the starting number for the first item (offset). After applying a template, update the values to ensure that the configuration complies with the API's requirements.

    Note: For licensing purposes, each request counts as one transaction, including each request for the next page of results.

    The value of the reserved, read-only getNextPage variable determines whether to request another page of results. As long as the getNextPage variable is true and the previous page contains data, the action continues to send requests for the next page.

    Note: You must explicitly set the value to true in the script or it will default to false.

    This is an optional Request component that only runs on the instance.

    Script step

    Run a script before every request for the next page of results. Use this script for data validation and transformation when calling a paginated API. For example, generate a JSON payload for the next page request. Selecting this option adds a script step to the Data Stream action. For more information, see Script step.

    This is an optional Request component that runs on either the instance or a MID Server.

    REST or SOAP step

    Send a REST or SOAP request to a third-party API. Select a data format to add an associated step to the Data Stream action. For more information, see REST step and SOAP step.

    This is a mandatory Request component that runs on either the instance or a MID Server.

    Parsing

    Use the Parsing category to configure how the action separates data stream elements into complex data objects. Use the Splitter step to identify and separate items from an XML or JSON stream, and use the Script Parser step to transform each item into a complex object. The Parsing section executes once per item in the stream.

    Splitting and parsing a stream of user records.

    For more information about complex data, see Complex data. Parsing components provide these configuration options.

    Splitter step

    Identify the parent node in the response stream to map to a complex object. For example, identify a user element in an XML payload to create a complex object for each user in the response stream.

    Select a splitter type to identify and separate repeated items in an XML or JSON data stream.

    • JSON: Identifies objects in a stream of JSON data.
    • XML: Identifies objects in an XML document.

    This is a mandatory Parsing component that only runs on the instance.

    Script Parser step

    Use JavaScript and ServiceNow APIs to map items in the response stream to a complex object output represented by the targetObject global object. For example, map incident record elements identified in the splitter step to a complex object containing incident fields. If the data stream includes siblings to the item identified in the splitter step that you do not want mapped to a complex object, include conditions to exclude those items.

    This is a mandatory Parsing component that only runs on the instance.

    Data Stream outputs

    When designing a Data Stream action, you must create a single output of type Object. The Script Parser step maps items in the stream to this object using the targetObject global object.

    At runtime, the system splits and parses the stream of response data according to the Data Stream configuration. Each item in the stream maps to the complex object structure defined by the Script Parser step and the object output, resulting in a large series of complex objects. For more information about complex data, see Complex data.

    Execution details

    View the configuration and runtime results for each item processed by a Data Stream action. Select a record number to see its configuration and runtime details. By default, the execution details include requests for the last 1000 items. To change the number of items in the execution details, update the com.snc.process_flow.reporting.datastream.item.lastn system property.

    Execution details for a Data Stream action.

    Data stream summary

    View an overview of the execution that includes this information.

    • Page count: Number of pages returned by a paginated API.
    • Total item count: Number of items in the response stream mapped to complex object outputs.
    • Error count: Number of errors encountered.
    Page details

    View runtime data for each step within the Data Stream action. Select a page to view runtime details for each request to a paginated API. By default, the execution details include requests for the last five pages. To change the number of requests in the execution details, update the com.snc.process_flow.page.reporting.lastn system property. Set the value to 0 to remove pages from the execution details and -1 to include all pages.

    Note: Including all pages can affect performance and is not recommended.

    Runtime details

    • Data Stream action design considerations

      Follow these considerations when creating a Data Stream action and when adding a Data Stream action to a flow to prevent errors and performance issues.

    • MID Server support for Data Stream actions

      Get data through a ServiceNow® MID Server when running a Data Stream action.

    • Create a Data Stream action

      Create a reusable action to process a stream of response data within a flow.

    • Test a Data Stream action

      Test your Data Stream action to ensure that it works the way you expect before you add it to a flow.

    • Use a Data Stream action in a flow

      Data Stream actions use the For each flow logic option to process stream data in a flow. For example, suppose that you want to use a Data Stream action that imports document data from a third-party site. When added to a flow, the action appears as a For each flow logic block, enabling flow designers to easily create a record in ServiceNow for each object in the data stream.

    Tags:

    Feedback
    On this page

    Previous topic

    Next topic

    • Contact Us
    • Careers
    • Terms of Use
    • Privacy Statement
    • Sitemap
    • © ServiceNow. All rights reserved.

    Release version
    Choose your release version

      Data Stream actions and pagination

      • Save as PDF Selected topic Topic & subtopics All topics in contents
      • Unsubscribe Log in to subscribe to topics and get notified when content changes.
      • Share this page

      Data Stream actions and pagination

      Send REST or SOAP requests from Flow Designer to APIs that return a stream of response data larger than 10 MB, or that return paginated results. Parse stream data into a series of complex object outputs and use the data pills in other actions in a flow.

      For example, create a Data Stream action to import a large quantity of employee data from a third-party HR site. The Data Stream action sends a REST request to the third-party site and processes the response to populate records in the User [sys_user] table.

      Benefits

      Data Stream actions offer these benefits.

      • Parse and format a stream of response data larger than 10 MB.
      • Automatically send multiple requests to APIs that paginate results.
      • Enable flow designers to process large requests without complex coding or configuration.
      • Enable flow designers to process each object within a data stream using For each flow logic. For example, you might create a Data Stream action that imports document data from a third-party site. When you add the action to a flow, Flow Designer automatically adds the action to a For each flow logic block, enabling flow designers to easily create a record in ServiceNow for each object in the data stream. See Use a Data Stream action in a flow.
      • Enable flow designers to reuse Data Stream actions in multiple flows, using the same source of data in multiple ways.

      Running a Data Stream action

      There are two ways to run a Data Stream action.

      From a flow
      You can process each object within a data stream using For each flow logic. For example, you might create a Data Stream action that imports document data from a third-party site. When you add the action to a flow, Flow Designer automatically adds the action to a For each flow logic block, enabling flow designers to easily create a record in ServiceNow for each object in the data stream. See Use a Data Stream action in a flow.
      From a script
      You can start a Data Stream using the executeDataStreamAction() method in the FlowAPI class. For more information, see FlowAPI.

      Action outline

      Data Stream actions follow a set structure. Follow prompts to add and remove steps from the action outline. You cannot manually add steps to a Data Stream action.

      Animation showing the Datastream Outline and selected options
      Note: Clearing an option in a configuration page removes the step from the Data Stream outline and deletes all data associated with the step.

      Action Preprocessing

      Use the Action Preprocessing category to run a preprocessing script before the action sends the initial API request. For example, validate action inputs or set default values. Preprocessing executes once per action, before the first API request.

      Selecting this option adds a script step to the Data Stream action. For more information, see Script step.

      This is an optional Data Stream action component that runs on either the instance or a MID Server.

      Request

      Use the Request category to configure how the action sends API requests. The Request section executes once per page of results. Request components provide these configuration options.

      Pagination Setup step

      Request results in batches. Once one page of data is processed, the Data Stream action runs the request section again to return the next set of results. Use the pagination setup step to set up pagination options required by the API. Configure the Pagination Setup step manually, or select a pre-built template to apply common configurations. For example, apply the Limit / Offset template to specify the number of items you want returned per page (limit), and the starting number for the first item (offset). After applying a template, update the values to ensure that the configuration complies with the API's requirements.

      Note: For licensing purposes, each request counts as one transaction, including each request for the next page of results.

      The value of the reserved, read-only getNextPage variable determines whether to request another page of results. As long as the getNextPage variable is true and the previous page contains data, the action continues to send requests for the next page.

      Note: You must explicitly set the value to true in the script or it will default to false.

      This is an optional Request component that only runs on the instance.

      Script step

      Run a script before every request for the next page of results. Use this script for data validation and transformation when calling a paginated API. For example, generate a JSON payload for the next page request. Selecting this option adds a script step to the Data Stream action. For more information, see Script step.

      This is an optional Request component that runs on either the instance or a MID Server.

      REST or SOAP step

      Send a REST or SOAP request to a third-party API. Select a data format to add an associated step to the Data Stream action. For more information, see REST step and SOAP step.

      This is a mandatory Request component that runs on either the instance or a MID Server.

      Parsing

      Use the Parsing category to configure how the action separates data stream elements into complex data objects. Use the Splitter step to identify and separate items from an XML or JSON stream, and use the Script Parser step to transform each item into a complex object. The Parsing section executes once per item in the stream.

      Splitting and parsing a stream of user records.

      For more information about complex data, see Complex data. Parsing components provide these configuration options.

      Splitter step

      Identify the parent node in the response stream to map to a complex object. For example, identify a user element in an XML payload to create a complex object for each user in the response stream.

      Select a splitter type to identify and separate repeated items in an XML or JSON data stream.

      • JSON: Identifies objects in a stream of JSON data.
      • XML: Identifies objects in an XML document.

      This is a mandatory Parsing component that only runs on the instance.

      Script Parser step

      Use JavaScript and ServiceNow APIs to map items in the response stream to a complex object output represented by the targetObject global object. For example, map incident record elements identified in the splitter step to a complex object containing incident fields. If the data stream includes siblings to the item identified in the splitter step that you do not want mapped to a complex object, include conditions to exclude those items.

      This is a mandatory Parsing component that only runs on the instance.

      Data Stream outputs

      When designing a Data Stream action, you must create a single output of type Object. The Script Parser step maps items in the stream to this object using the targetObject global object.

      At runtime, the system splits and parses the stream of response data according to the Data Stream configuration. Each item in the stream maps to the complex object structure defined by the Script Parser step and the object output, resulting in a large series of complex objects. For more information about complex data, see Complex data.

      Execution details

      View the configuration and runtime results for each item processed by a Data Stream action. Select a record number to see its configuration and runtime details. By default, the execution details include requests for the last 1000 items. To change the number of items in the execution details, update the com.snc.process_flow.reporting.datastream.item.lastn system property.

      Execution details for a Data Stream action.

      Data stream summary

      View an overview of the execution that includes this information.

      • Page count: Number of pages returned by a paginated API.
      • Total item count: Number of items in the response stream mapped to complex object outputs.
      • Error count: Number of errors encountered.
      Page details

      View runtime data for each step within the Data Stream action. Select a page to view runtime details for each request to a paginated API. By default, the execution details include requests for the last five pages. To change the number of requests in the execution details, update the com.snc.process_flow.page.reporting.lastn system property. Set the value to 0 to remove pages from the execution details and -1 to include all pages.

      Note: Including all pages can affect performance and is not recommended.

      Runtime details

      • Data Stream action design considerations

        Follow these considerations when creating a Data Stream action and when adding a Data Stream action to a flow to prevent errors and performance issues.

      • MID Server support for Data Stream actions

        Get data through a ServiceNow® MID Server when running a Data Stream action.

      • Create a Data Stream action

        Create a reusable action to process a stream of response data within a flow.

      • Test a Data Stream action

        Test your Data Stream action to ensure that it works the way you expect before you add it to a flow.

      • Use a Data Stream action in a flow

        Data Stream actions use the For each flow logic option to process stream data in a flow. For example, suppose that you want to use a Data Stream action that imports document data from a third-party site. When added to a flow, the action appears as a For each flow logic block, enabling flow designers to easily create a record in ServiceNow for each object in the data stream.

      Tags:

      Feedback

          Share this page

          Got it! Feel free to add a comment
          To share your product suggestions, visit the Idea Portal.
          Please let us know how to improve this content

          Check any that apply

          To share your product suggestions, visit the Idea Portal.
          Confirm

          We were unable to find "Coaching" in Jakarta. Would you like to search instead?

          No Yes
          • Contact Us
          • Careers
          • Terms of Use
          • Privacy Statement
          • Sitemap
          • © ServiceNow. All rights reserved.

          Subscribe Subscribed Unsubscribe Last updated: Tags: January February March April May June July August September October November December No Results Found Versions Search preferences successfully updated My release version successfully updated My release version successfully deleted An error has occurred. Please try again later. You have been unsubscribed from all topics. You are now subscribed to and will receive notifications if any changes are made to this page. You have been unsubscribed from this content Thank you for your feedback. Form temporarily unavailable. Please try again or contact  docfeedback@servicenow.com  to submit your comments. The topic you requested does not exist in the release. You were redirected to a related topic instead. The available release versions for this topic are listed There is no specific version for this documentation. Explore products Click to go to the page. Release notes and upgrades Click to open the dropdown menu. Delete Remove No selected version Reset This field is required You are already subscribed to this topic Attach screenshot The file you uploaded exceeds the allowed file size of 20MB. Please try again with a smaller file. Please complete the reCAPTCHA step to attach a screenshot
          Log in to personalize your search results and subscribe to topics
          No, thanks Login