IRE Architecture — The Engine That Keeps Your CMDB Clean

The Multi-Source Problem That IRE Solves

Modern IT environments have dozens of tools that each know something about your infrastructure. AWS Config knows your EC2 instances. VMware knows your virtual machines. Discovery scans the OS level. Manual import adds contract information. Each source has accurate data — but they don't always agree, and they don't know about each other.

Without a centralized intelligence layer, CMDB quality collapses:

• Duplicates: AWS says "i-0123abc" is a server at 10.0.1.5. Discovery also finds something at 10.0.1.5 and creates a separate CI. Now you have two CIs for the same machine.
• Data wars: AWS says cpu_count=8. Your old spreadsheet import says cpu_count=4. The last writer wins — and the last writer is whoever ran most recently, not whoever is most accurate.
• Phantom CIs: A server was decommissioned. The Import Set from the old CMDB still has a record for it. It keeps getting re-created.

IRE solves all three problems: it identifies existing CIs, reconciles conflicting field values using source authority rules, and detects duplicates before they are created.

IRE's Three Processing Phases

Every time data arrives at IRE — from Discovery, SGC, Import Set, or direct API call — it goes through three sequential phases. Understanding these phases is the foundation of everything else in Part 4.

Phase 1: Identification

IRE asks: "Does this CI already exist in the CMDB?"

It uses Identification Rules for the CI class to try to match the incoming CI payload against existing records. For example: "This incoming CI is a Linux server with serial number SN-12345. Does a CI with that serial number already exist in cmdb_ci_linux_server?"

• Match found: This is an update. Proceed to Reconciliation with the existing CI's sys_id.
• No match found: This is a new CI. IRE will create a new record. Skip Reconciliation and go directly to CMDB write.
• Multiple matches found: This is a potential duplicate situation. Proceed to Deduplication (Phase 3).

Phase 2: Reconciliation

IRE asks: "This CI already exists and multiple sources have values for its fields — which source's value should win?"

It uses Reconciliation Rules to determine source precedence for each field. For example: "The ip_address field is owned by Discovery. The owned_by field is owned by Manual Entry. The cost_center field is owned by the ITSM connector." Each source can update only the fields it owns.

A source that is not authoritative for a field cannot overwrite that field's value, even if it provides a different value. This prevents low-quality sources from overwriting data set by authoritative sources.

Phase 3: Deduplication Detection

IRE asks: "Are there existing CI records that represent the same physical or logical entity as each other?"

Deduplication runs as a separate process (or triggered by Phase 1 finding multiple matches). When potential duplicates are found, IRE can either:

• Automatically merge them according to a remediation playbook
• Flag them for manual review by creating a remediation task

Deduplication and Remediation Playbooks are covered in depth in Topic 4 of this Part.

The IRE API — How Data Enters IRE

IRE is not a UI feature — it's an internal API. Data ingestion systems (Discovery, SGC, Import Sets with IRE configuration) send structured payloads to the IRE API, which then processes them through the three phases.

The key tables involved in IRE API processing:

• cmdb_ci_payload_pending — Queue of incoming CI payloads awaiting processing
• cmdb_ire_event — Log of IRE processing events (success, failure, what happened to each CI)
• cmdb_ire_identification — Records of identification decisions made by IRE

What a CI Payload Contains

When Discovery or SGC sends data to IRE, it's structured as a payload containing:

• The CI class (e.g., cmdb_ci_linux_server)
• Field values (hostname, IP, serial number, OS version, etc.)
• The source label (which tool is sending this — "ServiceNow Discovery", "AWS SGC", etc.)
• Relationships to other CIs in the same payload (for batch processing)

Using the IRE API Directly

Developers can call the IRE API directly (without going through Discovery or SGC) using the ServiceNow scripted REST API or background scripts. This is useful for custom integrations that aren't served by existing SGCs. When you call the IRE API:

• All three IRE phases run exactly as they do for Discovery/SGC data
• Your integration benefits from deduplication and reconciliation automatically
• This is the correct way to build custom CMDB integrations — not direct Table API writes

IRE Sources and Source Labels

Every piece of data that IRE processes is tagged with a source label — the name of the ingestion tool that sent it. Source labels are the foundation of Reconciliation Rules.

How Source Labels Are Assigned

• Discovery: Automatically labeled "ServiceNow" (the platform's internal Discovery)
• SGC: Each connector has a unique source label (e.g., "ServiceNow AWS", "ServiceNow VMware", "ServiceNow GitHub")
• Import Sets with IRE: The source label is the Data Source record's name
• Custom IRE API calls: The source label is provided by the integration code

The Precedence Order (Out of the Box)

ServiceNow ships with a default source order that determines which source wins when multiple sources have values for the same field. You can customize this. Default order (high to low):

1. Manual Entry — A human explicitly set this value; highest trust
2. ServiceNow (Discovery) — Active scan found this value directly from the device
3. ServiceNow Service Graph Connectors — API-sourced data from cloud/monitoring providers
4. Orchestration — Values set by automated workflows
5. Third-party integrations — Custom Import Sets and external tools

This default order can be completely overridden per CI class and per field through Reconciliation Rules (Topic 3 of this Part).

IRE and CI Class Hierarchy

IRE processes CIs within the context of the CMDB class hierarchy (fully covered in Part 5). This is critical because Identification Rules and Reconciliation Rules are class-specific — the rules for cmdb_ci_linux_server are different from the rules for cmdb_ci_network_adapter.

A key IRE behavior: rules can be defined at a parent class and inherited by child classes, or overridden at the child class level. For example:

• A general Identification Rule for cmdb_ci_computer (parent of all computers) that matches on IP address
• A more specific Identification Rule for cmdb_ci_linux_server (child) that additionally checks serial number
• IRE uses the most specific matching rule — the Linux server rule wins over the generic computer rule

CI Class Determination by IRE

When incoming data says the CI class is cmdb_ci_linux_server, IRE enforces this class assignment. If the data contains field values that don't exist on cmdb_ci_linux_server, IRE ignores those fields. This prevents data quality issues from schema mismatches.

Monitoring IRE — The IRE Log

Every CI that passes through IRE generates a log entry. These logs are your primary diagnostic tool when Discovery or SGC produces incorrect or missing CMDB data.

Accessing the IRE Log

Navigate to Configuration → CMDB → Identification/Reconciliation or search for cmdb_ire_event.

Key Fields in IRE Log Records

Common IRE Log Error Messages

• "No identification rule found for class X" — IRE received a CI of class X but has no Identification Rule to match it against. Fix: create or enable an Identification Rule for class X.
• "Multiple CIs match identification criteria" — IRE found more than one existing CI matching the incoming data. Fix: investigate duplicate CIs using Deduplication tools.
• "CI payload missing required identification attributes" — The incoming payload doesn't contain the fields the Identification Rule needs. Fix: update the Discovery pattern or SGC mapping to include those fields.

The Multi-Source CMDB — IRE's Core Value

The concept of a Multi-Source CMDB is that multiple tools can each contribute their specialized data to the same CI record, and IRE manages whose data is authoritative for which fields.

This is a fundamental architectural shift from traditional CMDBs where one tool "owns" the CI. In a multi-source CMDB:

• AWS SGC creates the CI and sets cloud attributes (instance_type, ami_id, availability_zone)
• Discovery enriches the CI with OS attributes (kernel_version, installed_software)
• Service Mapping adds relationships (this CI is used by Application Service X)
• Manual entry sets business attributes (owned_by, cost_center, business_criticality)
• IRE Reconciliation Rules ensure each source only updates its authorized fields

IRE Performance and Processing

IRE processes CI payloads asynchronously via a queue. Understanding this helps you set correct expectations for when CMDB records appear after a Discovery or SGC run.

• Queue processing: After Discovery completes, CI payloads are queued. IRE processes them in order, not instantly. Large environments (10,000+ CIs in one run) may see a delay of minutes to hours before all records appear in the CMDB.
• Batch vs. single-CI API: Both IRE API modes exist. Batch is more efficient for bulk ingestion (Discovery, SGC); single-CI is for real-time API integrations.
• IRE queue monitoring: Under Configuration → CMDB → IRE Processing Queue, you can see pending payloads and their status. If the queue is growing without decreasing, IRE processing may be stalled.

Practice Questions

4 questions · Select an answer to see the explanation immediately.