The Automated Test Framework (ATF) enables
you to create and run automated tests to confirm that your instance works after
making a change. For example, after an upgrade, during application development,
or when deploying instance configurations with update sets. Review failed test
results to identify the changes that caused the failure and the changes that you
should review.
Note: By default, the system property to run automated tests is disabled to prevent you from
accidentally running them on a production system. Run tests only on development, test, and
other non-production instances to avoid data corruption and outage.
For developer training, see Using the Automated Test Framework on the
ServiceNow
Developer Site.
Benefits
Automated Test Framework provides these
benefits for change managers and developers.
- Reduce upgrade and development time by replacing manual testing with automated
testing.
- Design tests once and reuse them in different contexts and with different test data
sets.
- Keep test instances clean by rolling back test data and changes made after each test
run.
- Create test suites to organize and run tests in batches.
- Schedule test suite runs.
- Enable non-technical test designers to create tests of standard Now Platform functionality.
- Reduce test design time by copying quick start tests and test suites.
- Create custom test steps to expand test coverage.
Automated Test Framework records and
components
The Automated Test Framework consists of
these records and components.
Test
A test is a logical grouping of related automated test steps that verify some
functionality or feature. Each test is a record in the Test [sys_atf_test] table. Test
designers typically create a test to verify one feature or a group of related features. For
example, the CSM: Create Product Case test validates the creation of
Product Case records. Each test has a related list of test steps and test results.
Test suite
A test suite is a collection of tests that run in a specific order. Test
designers typically create a test suite to test an application or a group of related
features. For example, the CSM: Case Management test suite validates
the functionality of the Customer Service Management application. Test designers can
schedule running test suites and starting any required client test runners.
Quick start test
A quick start test is a ServiceNow-provided test or test suite
installed with the demo data of an application. Use quick start tests as templates to build
your own tests and test suites.
Test step
A test step combines a step configuration with the runtime test data necessary
to run a step. The test step always specifies the order in which it runs in the test. Test
steps have their own related list of step results. Each test step is a record in the Test
Step [sys_atf_step] table that specifies a test action, the step configuration, and an
execution order. Test designers add test steps to tests to verify functionality. For
example, the first test step of the CSM: Create Product Case test is
to impersonate the demo user John Jason who is authorized for Case Management.
Step configuration
A step configuration is a specific test action the Automated Test Framework can run. Step
configurations do not contain any runtime test data and can only be run when test designers
add them as part of a test step. Each step configuration is a record in the Test Step Config
[sys_atf_step_config] table that specifies the input variables used to run the step
configuration and the output variables available to other step configurations. For example,
the Impersonate step configuration allows a test to impersonate
another user.
The Automated Test Framework provides a
default list of step configurations for most use cases and allows test designers to create
their own custom step configurations.
Step variable
A step variable stores step-specific input and output values. For example, the
Open a New Form step configuration has variables to specify the
table and form view names. Use step variables to specify a particular test step target or to
pass information to other test steps.
Test result
A test result stores the output of a test or test suite run. Each test result
is a record in the Test Result [sys_atf_test_result] table that specifies the duration of
the test run, the status of the test, and screenshots where available. Use test results to
identify failing or non-running tests, and use the test logs to see more information about
test results. By default, the system deletes test and test suite results 30 days after
creation unless you enable the option to retain the test result indefinitely.
Step result
A step result stores the output of a test step run. Each step result is a
record in the Step Result [sys_atf_test_result_step] table that specifies the status of the
test step, a summary of the output, and a complete log of the output generated by the test
step. Use step results to identify failures and functionality needing review.
Assert type
The Assert type field specifies the conditions that must be met for
a test to pass. Test designers can use assertions to specify whether the results of an
operation are expected or unexpected. For example, suppose you want to test that a record
cannot be updated. In this case, you would add a Record Insert test step and set the
Assert type field to Record was not
inserted. The test passes when the record insert fails.
Some test step categories that include steps with an Assert type
field include:
- Server category: Assert which CRUD
operations cause a test to pass or fail.
- Custom UI category: Assert which component
states cause a test to pass or fail, and whether visible text causes a test to pass or
fail.
- Forms in Service Portal category: Assert whether a
form canceled in the browser due to validation errors, or a form successfully submitted
to the server causes a test to pass.
Client test runners
A client test runner is a browser tab that runs client-side test steps within
a Now Platform user interface. Client
test runners require a browser tab to function. If no client test runner is available when
you run a test, the system prompts you to open one. Testers can manually start a client test
runner or select an existing client test runner. Test designers can schedule starting client
test runners when they schedule running a test suite.
User roles
Assign roles to define Automated Test Framework permissions.
| Role Title [name] |
Role Description |
| atf_test_admin |
Create or edit Automated Test Framework properties. Has permission to:
- View the tests page
- Create/edit/delete tests
- Create/edit/delete test steps
- View the step config page
- View the test runner page
- View the test suite results, test results and result items pages
- Execute user tests
- View, create, edit, delete and execute test suites
- Create/edit step config records
- Create/edit Automated Test Framework properties
|
| atf_test_designer |
View Automated Test Framework properties only (cannot create or edit
properties). Has permission to:
- View the tests page
- Create/edit/delete tests
- Create/edit/delete test steps
- View the step config page
- View the test runner page
- View the test suite results, test results and result items pages
- Execute user tests
- View, create, edit, delete and execute test suites
- View Automated Test Framework properties
|
