Product documentation Docs
    • English
    • Deutsch
    • 日本語
    • 한국어
    • Français
  • More Sites
    • Now Community
    • Developer Site
    • Knowledge Base
    • Product Information
    • ServiceNow.com
    • Training
    • Customer Success Center
    • ServiceNow Support Videos
  • Log in

Product documentation

  • Home
How search works:
  • Punctuation and capital letters are ignored
  • Special characters like underscores (_) are removed
  • Known synonyms are applied
  • The most relevant topics (based on weighting and matching to search terms) are listed first in search results
Topics are ranked in search results by how closely they match your search terms
  • A match on the entire phrase you typed
  • A match on part of the phrase you typed
  • A match on ALL of the terms in the phrase you typed
  • A match on ANY of the terms in the phrase you typed

Note: Matches in titles are always highly ranked.

  • Release version
    Table of Contents
    • DevOps
Table of Contents
Choose your release version
    Home Orlando DevOps DevOps DevOps User-created DevOps integrations DevOps test tool integration

    DevOps test tool integration

    • Save as PDF Selected topic Topic & subtopics All topics in contents
    • Unsubscribe Log in to subscribe to topics and get notified when content changes.
    • Share this page

    DevOps test tool integration

    Test tool integration lets you view test results in DevOps for Jenkins, Azure DevOps, and GitLab unit, functional, and performance tests.

    Starting with version 1.15, JUnit test type integration is supported for Jenkins and Azure DevOps.

    Starting with version 1.23, JUnit test type integration is supported for GitLab.

    Note: For other test types, use the DevOps - POST /devops/tool/{capability} endpoint of the DevOps API.
    Starting with version 1.20:
    • Selenium tests run and published using TestNG are reported by the Jenkins plugin for ServiceNow DevOps.
    • Test type categorization is supported.
    • Additional tests results reported by tools, such as Apache JMeter, can be processed in DevOps using custom Flow Designer subflows (no pipeline changes required).
    Category Test type
    Unit

    JUnit (default)

    You can change the default test type by modifying the [sn_devops.default_test_type] DevOps property.

    Functional
    • Integration
    • Regression
    • Smoke
    • System
    • User Acceptance
    Performance Load

    Test type mapping

    The test type mapping connects the test type and entity being tested with the DevOps tool (DevOps > Integrations > Test Type Mappings module.)

    An accurate test type mapping ensures that the test type always appears as intended in the test summary results.

    Field Description
    Test type
    • JUnit
    • Integration
    • Regression
    • Smoke
    • System
    • User Acceptance
    • Load
    DevOps Entity Id Table name

    DevOps table name that contains the entity linked to the test results (in the test report payload).

    • Step [sn_devops_step]
    • Pipeline [sn_devops_pipeline]
    Note: Only DevOps Step and Pipeline tables are supported.
    Document

    Name of the entity specified in the selected table.

    For example, the name of the step, pipeline, artifact, or package.

    Test File Paths

    (Jenkins tests only)

    Path to the generated test result file on the Jenkins server.

    This is useful so test reports with attributes that don't conform to JUnit or TestNG implementation, such as JMeter for example, can still be leveraged by DevOps.

    Separate multiple files by a comma.

    Note: You must use a Flow Designer subflow to transform a raw test payload.
    Tool integration

    Tool that's running the test.

    DevOps Table

    DevOps table that corresponds to the table name in the DevOps Entity Id setting.

    Figure 1. DevOps test type mapping
    DevOps test type mapping

    Transforming a raw test payload

    You can transform a raw test report (a report containing attributes that do not conform to JUnit or TestNG implementation), such as JMeter for example, by configuring the DevOps Test Subflow Policy decision table to call a custom subflow (Decision Tables > Decision Tables module).
    Note: You must create the custom Flow Designer subflow that transforms the raw test payload.

    If there is more than one test type in a performance stage, you can use the DevOps Test Type Policy decision table to configure the test type for each test so the test result payloads are transformed correctly.

    Figure 2. DevOps decision tables
    DevOps decision tables
    Table 1. DevOps decision tables
    Decision table Purpose Configuration
    DevOps Test Subflow Policy

    To automatically call a custom subflow that transforms the raw payload received by the tool.

    Decision inputs:

    • Test Result Payload
    • Test Type

    Create a decision that specifies the custom subflow to call when the raw payload is received.

    Set the conditions to contain fields that would be included in the raw payload.

    For example, to call Jenkins BZ Performance Test custom subflow:

    Conditions:
    • Test Type is Load

      (Load is the test type configured for performance tests)

    • Test Result Payload contains throughput
    • Test Result Payload contains concurrency

    Answer: FLow: Jenkins BZ Performance Test

    DevOps Test Type Policy

    To automatically set test types when more than one type of test is configured in a performance test stage.

    This is necessary so the results for the second test type get transformed correctly.

    For example, when both a Load performance test and a JUnit performance test are mapped in the same DevOps step, the JUnit test results won't get formatted correctly unless a decision is created.

    Decision inputs:
    • Step
    • Test Result Payload
    • Tool Integration
    • Pipeline

    Create a decision for each type of test in the performance test stage to set the test type.

    Load test:
    • Conditions:
      • Step is Perf Tests
      • Test Result Payload contains throughput
      • Test Result Payload contains concurrency
    • Answer: TestType: Load

    JUnit test:

    • Conditions:
      • Step is Perf Tests
      • Test Result Payload does not contain throughput
      • Test Result Payload does not contain concurrency
    • Answer: TestType: JUnit

    Figure 3. DevOps multiple performance test types
    DevOps multiple performance test types
    Figure 4. DevOps multiple test type mappings
    DevOps test type mappings
    Figure 5. DevOps decision table decision
    DevOps decision table decision

    Test summary results

    You can view test summary results these ways.
    • DevOps > Test Results module (Test Summaries and Performance Test Summaries).
    • DevOps change request - Test Results related list.
    • DevOps Pipeline UI - Quality tile.
    Figure 6. DevOps performance test summary example
    DevOps performance test summary

    Expected standard JSON Notification capability payload - Test tool

    Functional:
    { 
    "name": "CorpSite-selenium#55", 
    "duration": 78.802, 
    "passedTests": 4, 
    "failedTests": 0, 
    "skippedTests": 0, 
    "blockedTests": 0, 
    "totalTests": 4, 
    "startTime": "2020-06-30T18:12:31Z", 
    "finishTime": "2020-06-30T18:12:31Z", 
    "passingPercent": 100, 
     
     
    // Use Artifact OR Package OR Build + Stage + PipelineName Attributes 
    "packages": [{"name": "CorpSite-pkg1"}], 
    "artifacts": [{"name": "CorpSite-artifact", "version": "1.0.0"}], 
    "buildNumber": "55", 
    "stageName": "test", 
    "pipelineName": "CorpSite-selenium", 
    } 
    
    Performance:
    { 
    "name": "Performance Tests", 
    "url": "http://abc.com", 
    "startTime": "2020-06-30T18:12:31Z", 
    "finishTime": "2020-06-30T18:12:31Z", 
    "duration": 78.802, 
    "maximumVirtualUsers": "", 
    "throughput": "", 
    "maximumTime": "", 
    "minimumTime": "", 
    "averageTime": "", 
    "ninetyPercent": "", 
    "standardDeviation": "", 
     
    // Use Artifact OR Package OR Build + Stage + PipelineName Attributes 
    "packages": [{"name": "CorpSite-pkg1"}], 
    "artifacts": [{"name": "CorpSite-artifact", "version": "1.0.0"}], 
    "buildNumber": "55", 
    "stageName": "test", 
    "pipelineName": "CorpSite-Performance", 
    } 
    
    • Configure a test tool in DevOps

      Configure a test tool in DevOps to view unit, functional, and performance test results.

    Tags:

    Feedback
    On this page

    Previous topic

    Next topic

    • Contact Us
    • Careers
    • Terms of Use
    • Privacy Statement
    • Sitemap
    • © ServiceNow. All rights reserved.

    Release version
    Choose your release version

      DevOps test tool integration

      • Save as PDF Selected topic Topic & subtopics All topics in contents
      • Unsubscribe Log in to subscribe to topics and get notified when content changes.
      • Share this page

      DevOps test tool integration

      Test tool integration lets you view test results in DevOps for Jenkins, Azure DevOps, and GitLab unit, functional, and performance tests.

      Starting with version 1.15, JUnit test type integration is supported for Jenkins and Azure DevOps.

      Starting with version 1.23, JUnit test type integration is supported for GitLab.

      Note: For other test types, use the DevOps - POST /devops/tool/{capability} endpoint of the DevOps API.
      Starting with version 1.20:
      • Selenium tests run and published using TestNG are reported by the Jenkins plugin for ServiceNow DevOps.
      • Test type categorization is supported.
      • Additional tests results reported by tools, such as Apache JMeter, can be processed in DevOps using custom Flow Designer subflows (no pipeline changes required).
      Category Test type
      Unit

      JUnit (default)

      You can change the default test type by modifying the [sn_devops.default_test_type] DevOps property.

      Functional
      • Integration
      • Regression
      • Smoke
      • System
      • User Acceptance
      Performance Load

      Test type mapping

      The test type mapping connects the test type and entity being tested with the DevOps tool (DevOps > Integrations > Test Type Mappings module.)

      An accurate test type mapping ensures that the test type always appears as intended in the test summary results.

      Field Description
      Test type
      • JUnit
      • Integration
      • Regression
      • Smoke
      • System
      • User Acceptance
      • Load
      DevOps Entity Id Table name

      DevOps table name that contains the entity linked to the test results (in the test report payload).

      • Step [sn_devops_step]
      • Pipeline [sn_devops_pipeline]
      Note: Only DevOps Step and Pipeline tables are supported.
      Document

      Name of the entity specified in the selected table.

      For example, the name of the step, pipeline, artifact, or package.

      Test File Paths

      (Jenkins tests only)

      Path to the generated test result file on the Jenkins server.

      This is useful so test reports with attributes that don't conform to JUnit or TestNG implementation, such as JMeter for example, can still be leveraged by DevOps.

      Separate multiple files by a comma.

      Note: You must use a Flow Designer subflow to transform a raw test payload.
      Tool integration

      Tool that's running the test.

      DevOps Table

      DevOps table that corresponds to the table name in the DevOps Entity Id setting.

      Figure 1. DevOps test type mapping
      DevOps test type mapping

      Transforming a raw test payload

      You can transform a raw test report (a report containing attributes that do not conform to JUnit or TestNG implementation), such as JMeter for example, by configuring the DevOps Test Subflow Policy decision table to call a custom subflow (Decision Tables > Decision Tables module).
      Note: You must create the custom Flow Designer subflow that transforms the raw test payload.

      If there is more than one test type in a performance stage, you can use the DevOps Test Type Policy decision table to configure the test type for each test so the test result payloads are transformed correctly.

      Figure 2. DevOps decision tables
      DevOps decision tables
      Table 1. DevOps decision tables
      Decision table Purpose Configuration
      DevOps Test Subflow Policy

      To automatically call a custom subflow that transforms the raw payload received by the tool.

      Decision inputs:

      • Test Result Payload
      • Test Type

      Create a decision that specifies the custom subflow to call when the raw payload is received.

      Set the conditions to contain fields that would be included in the raw payload.

      For example, to call Jenkins BZ Performance Test custom subflow:

      Conditions:
      • Test Type is Load

        (Load is the test type configured for performance tests)

      • Test Result Payload contains throughput
      • Test Result Payload contains concurrency

      Answer: FLow: Jenkins BZ Performance Test

      DevOps Test Type Policy

      To automatically set test types when more than one type of test is configured in a performance test stage.

      This is necessary so the results for the second test type get transformed correctly.

      For example, when both a Load performance test and a JUnit performance test are mapped in the same DevOps step, the JUnit test results won't get formatted correctly unless a decision is created.

      Decision inputs:
      • Step
      • Test Result Payload
      • Tool Integration
      • Pipeline

      Create a decision for each type of test in the performance test stage to set the test type.

      Load test:
      • Conditions:
        • Step is Perf Tests
        • Test Result Payload contains throughput
        • Test Result Payload contains concurrency
      • Answer: TestType: Load

      JUnit test:

      • Conditions:
        • Step is Perf Tests
        • Test Result Payload does not contain throughput
        • Test Result Payload does not contain concurrency
      • Answer: TestType: JUnit

      Figure 3. DevOps multiple performance test types
      DevOps multiple performance test types
      Figure 4. DevOps multiple test type mappings
      DevOps test type mappings
      Figure 5. DevOps decision table decision
      DevOps decision table decision

      Test summary results

      You can view test summary results these ways.
      • DevOps > Test Results module (Test Summaries and Performance Test Summaries).
      • DevOps change request - Test Results related list.
      • DevOps Pipeline UI - Quality tile.
      Figure 6. DevOps performance test summary example
      DevOps performance test summary

      Expected standard JSON Notification capability payload - Test tool

      Functional:
      { 
      "name": "CorpSite-selenium#55", 
      "duration": 78.802, 
      "passedTests": 4, 
      "failedTests": 0, 
      "skippedTests": 0, 
      "blockedTests": 0, 
      "totalTests": 4, 
      "startTime": "2020-06-30T18:12:31Z", 
      "finishTime": "2020-06-30T18:12:31Z", 
      "passingPercent": 100, 
       
       
      // Use Artifact OR Package OR Build + Stage + PipelineName Attributes 
      "packages": [{"name": "CorpSite-pkg1"}], 
      "artifacts": [{"name": "CorpSite-artifact", "version": "1.0.0"}], 
      "buildNumber": "55", 
      "stageName": "test", 
      "pipelineName": "CorpSite-selenium", 
      } 
      
      Performance:
      { 
      "name": "Performance Tests", 
      "url": "http://abc.com", 
      "startTime": "2020-06-30T18:12:31Z", 
      "finishTime": "2020-06-30T18:12:31Z", 
      "duration": 78.802, 
      "maximumVirtualUsers": "", 
      "throughput": "", 
      "maximumTime": "", 
      "minimumTime": "", 
      "averageTime": "", 
      "ninetyPercent": "", 
      "standardDeviation": "", 
       
      // Use Artifact OR Package OR Build + Stage + PipelineName Attributes 
      "packages": [{"name": "CorpSite-pkg1"}], 
      "artifacts": [{"name": "CorpSite-artifact", "version": "1.0.0"}], 
      "buildNumber": "55", 
      "stageName": "test", 
      "pipelineName": "CorpSite-Performance", 
      } 
      
      • Configure a test tool in DevOps

        Configure a test tool in DevOps to view unit, functional, and performance test results.

      Tags:

      Feedback

          Share this page

          Got it! Feel free to add a comment
          To share your product suggestions, visit the Idea Portal.
          Please let us know how to improve this content

          Check any that apply

          To share your product suggestions, visit the Idea Portal.
          Confirm

          We were unable to find "Coaching" in Jakarta. Would you like to search instead?

          No Yes
          • Contact Us
          • Careers
          • Terms of Use
          • Privacy Statement
          • Sitemap
          • © ServiceNow. All rights reserved.

          Subscribe Subscribed Unsubscribe Last updated: Tags: January February March April May June July August September October November December No Results Found Versions Search preferences successfully updated My release version successfully updated My release version successfully deleted An error has occurred. Please try again later. You have been unsubscribed from all topics. You are now subscribed to and will receive notifications if any changes are made to this page. You have been unsubscribed from this content Thank you for your feedback. Form temporarily unavailable. Please try again or contact  docfeedback@servicenow.com  to submit your comments. The topic you requested does not exist in the release. You were redirected to a related topic instead. The available release versions for this topic are listed There is no specific version for this documentation. Explore products Click to go to the page. Release notes and upgrades Click to open the dropdown menu. Delete Remove No selected version Reset This field is required You are already subscribed to this topic Attach screenshot The file you uploaded exceeds the allowed file size of 20MB. Please try again with a smaller file. Please complete the reCAPTCHA step to attach a screenshot
          Log in to personalize your search results and subscribe to topics
          No, thanks Login