Thank you for your feedback.
Form temporarily unavailable. Please try again or contact docfeedback@servicenow.com to submit your comments.

Remediate a de-duplication task

Log in to subscribe to topics and get notified when content changes.

Remediate a de-duplication task

Remediate a de-duplication task by using the Duplicate CI Remediator wizard. Use the wizard to guide you through the duplicate CI reconciliation process or to apply a custom workflow.

Before you begin

Important concepts and elements associated with the Duplicate CI Remediator:
Master CI
The master CI is one of the duplicate CIs that you want to retain as an active CI while potentially retiring or deleting the rest of the duplicate CIs. The first step in the Duplicate CI Remediator is to select a master CI for the remediation process. The Duplicate CI Remediator lets you choose which attribute values, relationships, and related items from the duplicate CIs to reconcile into the master CI. Alternatively, you can choose not to consolidate any data and retain the master CI as it is.
Default related items list
A list of related items that is used globally in the Duplicate CI Remediator for all de-duplication tasks. All items from the default related items list are selected by default to be merged to the master CI, on the Merge Relationships and Related Items tab. Adding or removing items from that slushbucket does not affect the default related items list. See Manage default related items list for more information.
Note: Asset related tables are not included in the default related items list and therefore they are not available for merge.
Properties related to CI de-duplciation:
  • glide.duplicate_ci_remediator.dry_run

    This system property controls whether the Duplicate CI Remediator actually remediates CI duplication by updating records in the CMDB, or not. When set to false (default value), updates specified in the wizard are actually performed. You can set this property to true and then test run through the Duplicate CI Remediator without any records actually being updated. In this case, the work notes for the task describe the changes that will happen in an actual remediation.

  • glide.duplicate_ci_remediator.max.field_length

    Attributes in which max_length exceeds this property value (4000 by default) are excluded from the Select Master CI, Merge Attribute Values, and Determine Duplicate CI Actions tabs in the Duplicate CI Remediator wizard. However, if the max_length for an attribute is equal to the property value, and the size of the data exceeds the property value, then the data is truncated to the property value and the attribute appears in attribute lists. This property impacts the performance of de-duplication tasks, therefore be cautious about setting this value.

  • glide.duplicate_ci_remediator.related_items_blacklist

    Comma separated list of related tables in the format '<table>.<reference column>', that are excluded from merging during duplicate CI remediation.

Role required: itil_admin

About this task

As you progress through the tabs of the Duplicate CI Remediator, CIs are not updated. All updates are applied only in the final step, after you click Remediate.

Note:
  • Merging of attributes and related items that are associated with assets is not supported in the Duplicate CI Remediator.
  • The Duplicate CI Remediator behaves differently in remediations that involve a large number of duplicate CIs or where the duplicates are serial numbers. For information about those special cases, see the 'Special remediation scenarios' section at the bottom.

Procedure

  1. Navigate to Configuration > Identification/Reconciliation > De-duplication Tasks.
  2. Open the de-duplication task that you want to remediate. On the task form, click Remediate.
  3. In the Remediate dialog box, select either of the following options:
    • Use the Duplicate CI Remediator (Recommended): Use the wizard to consolidate duplicate CIs according to your configurations and settings. Follow the Duplicate CI Remediator tabs to configure the reconciliation.
    • Use a custom remediation workflow: Use an existing CMDB remediation rule or select Add New to create a new one.
      • On the CMDB Remediation Rule form, set Task type to Remediate Duplicate Task and select Active.
      • On the Workflow form, set Table to Remediate Duplicate Task [reconcile_duplicate_task] and If condition matches to None.
      • Ensure that the associated workflow remediates duplicate CIs.
      In the Remediate dialog box, click Next to start the workflow and to exit the Duplicate CI Remediator. The Remediate Duplicate Task form appears, where you can update the State of the task.
  4. On the Select Master CI tab in the Duplicate CI Remediator:
    1. Select the master CI for this reconciliation using either of the following lists of duplicate CIs. For any CI, you can click the Name link to display the CI's attributes, or click the Related Items link to display the number of related items.
      • Recommended: A subset of the All list, containing only system recommended master CIs. System recommendations are based on checking the duplicate CIs for the following criteria:
        • CI with most related items.
        • CI with most relationships.
        • Newest discovered CI.
        • Newest updated CI.
        • Oldest created CI.
        • Previous master CI, if one was previously selected.
      • All: All duplicate CIs for the de-duplication task.

      Either option lets you review a summary of the remediation settings before starting the remediation.

    2. Select one of the following options to choose whether to consolidate any attribute values, relationships, or related items from any of the duplicate CIs into the master CI:
      • Remediate Manually: Lets you specify which attribute values, relationships, and related items from duplicate CIs to consolidate into the master CI.
      • Use Master CI: Retains master CI attribute values, merges relationships, and merges only the default related items.

        Skip to step number 7 as this selection skips all configurations other than choosing the action for the duplicate CIs on the Determine Duplicate CI Actions page.

  5. On the Merge Attribute Values tab review inconsistent values of each attribute. For each attribute, choose to retain the master CI's value, or choose a value from a duplicate CI for the master CI to be set with. Then click Next.
    Column Description
    Attribute Attribute for which there are different values.
    Master CI Value Attribute value in the master CI.
    Other Values The number of unique Attribute values within the duplicate CIs.

    To override the master CI attribute value with a duplicate CI value:

    1. Click the Other Values link.
    2. In the attribute dialog box, click Unique Values to display only unique attribute values, or All to display all attribute values including identical values.
    3. Select a value for the master CI Attribute to be set with.
      You can click Reset to Original to undo the selection of a different attribute value for the master CI.
    4. Click Select.
    Note: Attributes, such as system fields, discovery fields (discovery_source, last_discovered, first_discovered), and read-only fields (such as the asset field) do not appear in the list.
  6. On the Merge Relationships and Related Items tab:
    1. In the Merge Relationships section, select whether to merge all relationships from all duplicate CIs into the master CI. Click View all relationships to display all the relationships in which a duplicate CI is a parent or a child. You can click a Parent or a Child link to display more details. Orphan and duplicate relationships of duplicate CIs are deleted if you choose to merge relationships. For more information about the CMDB Health relationship KPI, see CMDB Health KPIs and metrics.
    2. In the Merge Related Items section use the slushbucket to select related items to be merged into the master CI. Click View all related items to display all related tables (items) and the count of references in each table to one of the duplicate CIs. You can click the links in Master CI Related Items and Duplicate CIs Related Items to display details about the related items.

      Related items in the slushbucket have the following format:Details of related items in thee slushbucket.

      • By default, all items in the default related items list are selected to be merged.
      • Related items (tables) that have no references to a duplicate CI are not listed, unless that table is included in the default related items list.
      • Since asset related tables are part of the blacklist, they are not available for merge.

      See Manage default related items list for more information about configuring a default list of related items.

    3. Click Next.
  7. On the Determine Duplicate CI Actions tab, choose one of the following actions to perform after completing the reconciliation. Then click Next.
    • Set attributes to custom values (recommended): Retain all duplicate CIs. Mark the duplicate CIs as invalid by setting a specific Attribute to a specific Value for all duplicate CIs. For example, set Operational Status to Retired to retire the duplicate CIs.
      If the identification engine is configured to skip duplication, then the discovery source of the duplicate CIs is automatically set to Duplicate and the duplicate CIs will not be added to any de-duplication task after this task is remediated. If skip duplication is not configured, ensure that identification inclusion rules are configured to exclude the duplicate CIs during identification. This configuration prevents new de-duplication tasks with the same duplicate CIs from being created after remediation.
      Note: Discovery fields, system fields, read only fields, and date fields are excluded from the attributes list.
    • Delete: Delete all duplicate CIs (only the master CI remains).
      Note: Review Roll back and delete recovery for information about reverting the deletion of CIs and related records.
  8. On the Review and Confirm tab:
    1. Review the summary of the expected updates for this duplicate CIs reconciliation. Updates are based on your selections and therefore the summary includes only the details that are applicable. This summary can include details of the relationships and related items that will be merged to the master CI, the attribute values that the master CI will be set with, and the number of CIs that will be deleted. Click Attributes, Relationships, Related Items, or Duplicate CI Actions if applicable, to display further details such as changes to attribute values.
    2. Click Remediate to complete the reconciliation according to your reconciliation settings.
      Once complete, the task State is set to Closed Complete.

Result

The following relationships are deleted without being merged to the master CI:
  • Relationships in which the type, child, or parent field is empty.
  • Relationships for which merging to the master CI will result in cyclic relationships.
  • Relationship for which merging to the master CI will result in duplicate relationships.

What to do next

The reconciliation process runs in the background and may take a while to complete. Upon completion, the system sends a confirmation notice to the remediator of the task. Meanwhile, you can:
  • Review Identification Rules: Review identification and inclusion rules and make any necessary updates to reduce CI duplication.
  • Check Progress: View the task activities that are logged as the remediation progresses.
  • View Master CI: View the master CI for this reconciliation process.

Special remediation scenarios

There are a few special remediation scenarios in which the Duplicate CI Remediator behaves differently.

Large number of duplicate CIs

Support for reconciliation of duplicate CIs in the Duplicate CI Remediator is limited when the number of duplicate CIs exceeds a certain threshold. This threshold is based on the value of the glide.duplicate_ci_remediator.max.cis property, which is 1,000 by default. You can update this property to increase the threshold. However, this threshold never exceeds 5,000, even if you set the property to a value greater than 5,000.

When the number of duplicate CIs for a de-duplication task exceeds the threshold, the options available in the wizard are limited:
  • On the Select Master CI tab, only the Recommended list of master CIs appears, and only the Use Master CI option is available.
  • Recommendations are based only on the oldest created, newest updated, and most recently discovered CIs.
  • Reconciliation of attribute conflicts and CI relationships is not supported, and only default related items are reconciled.
Duplicate serial numbers:
The Duplicate CI Remediator is usually applied to duplicate CMDB CIs. However, in some situations de-duplication tasks might be created for duplicate serial numbers. When the Duplicate CI Remediator processes duplicate serial numbers, the merge of relationships from duplicate records, is not referenced and is not applied.
Feedback