› CIS-DF Hub › Part 2 › Import Sets Architecture
⬡ Part 2 · Topic 1

Import Sets Architecture

Import Sets are ServiceNow's structured data ingestion pipeline. Learn the staging-then-transform architecture, why it exists, the Service Graph Connector as the CMDB-preferred path, and the Import Set API for programmatic ingestion.

📋 6 sections ⏱ ~25 min read 🎯 ~14% exam weight 🏷 Data Migration
🔗
Builds On — Part 1 Data Model Import Sets write data into ServiceNow's live tables — the same tables you studied in Part 1. You need table inheritance (Part 1 Topic 1) to understand which target table to write a CI into (cmdb_ci_server vs cmdb_ci_computer), and field types and the dictionary (Part 1 Topic 2) to understand why Transform Maps must coerce data types before writing. The staging table is temporary; the target table is a real Part 1 table in the live CMDB.
Section 1

Why Import Sets Exist

When external data arrives into ServiceNow — whether from a CSV file, a database, an API, or a discovery scanner — you almost never want to write it directly into your live tables. Why? Because external data is messy: column names don't match, values are formatted differently, duplicate records exist, and some fields need transformation before they make sense in ServiceNow.

Import Sets solve this by creating a two-stage pipeline: first, load raw data into a temporary staging table (the "Import Set table"), then run a Transform Map to clean, map, and push the data into the target live table. If something goes wrong, you fix the transformation logic and re-run — the raw data is still in the staging table, untouched.

💡
Analogy — Customs Inspection at the Border Think of Import Sets like customs at an international airport. When cargo (data) arrives from outside, it goes to a holding area (staging table) for inspection and processing. Inspectors (Transform Maps) check every item, apply rules, and either pass it through to the destination (target table) or flag it for review. Nothing goes directly to its final destination without going through customs first.
Section 2

The Import Set Pipeline — Step by Step

Understanding the full pipeline is essential — the exam tests each step independently.

Data Source
Step 1
›
Import Set Table
Step 2 — Staging
›
Transform Map
Step 3
›
Target Table
Step 4 — CMDB / Any

Step 1 — Data Source

A Data Source record defines where data comes from and how to retrieve it. Types include: File (CSV/Excel/XML uploaded or retrieved via SFTP), JDBC (direct database query), REST (fetch JSON/XML from an API), LDAP, and more. The Data Source is configured in System Import Sets → Data Sources.

Step 2 — Import Set Table (Staging)

When a Data Source is loaded, ServiceNow creates a temporary staging table — the Import Set table. This table has one column for each column in the source data. All rows are stored here first, untransformed, in their raw form. The table is named with the pattern u_imp_[something] or a name you specify.

Each load run creates one sys_import_set header record (the "Import Set") and multiple sys_import_set_row records — one per row of source data. These are what you see in the Import Set list view.

Step 3 — Transform Map

A Transform Map defines how columns in the staging table map to fields in the target table, and what transformation logic to apply. It runs after the staging table is populated. We cover Transform Maps in depth in Topic 2.

Step 4 — Target Table

The final destination. For non-CMDB data, records are inserted or updated in the target table based on the Transform Map's coalesce and field map settings. For CMDB data, records go through the IRE (Identification and Reconciliation Engine) before landing in the cmdb_ci hierarchy.

⚠
Exam Trap — Staging Table Cleanup Import Set staging tables grow unboundedly unless you clean them up. After successful transforms, the staging rows still exist. Large staging tables slow down the import process over time. Always configure a Data Retention Policy or scheduled job to clean old Import Set rows. The exam may ask about performance issues caused by uncleaned staging tables.
Section 3

Service Graph Connectors — The CMDB-Preferred Import Path

For CMDB data specifically, ServiceNow recommends using Service Graph Connectors rather than building custom Import Set pipelines. A Service Graph Connector is a certified, pre-built integration that loads CI data from a specific source system (AWS, Azure, Qualys, SCCM, etc.) into the CMDB via the IRE.

Why are they preferred? Three reasons:

  1. IRE integration — Service Graph Connectors feed data through the IRE automatically, ensuring proper duplicate detection, class identification, and reconciliation. A raw Import Set transform that inserts directly to cmdb_ci bypasses IRE entirely.
  2. Payload format — Connectors use the CMDB API's structured payload format, which carries CI class information, identification attributes, and relationship data together — not just flat field values.
  3. Source attribution — The IRE needs to know which "data source" provided which CI data so it can apply Reconciliation Rules (which source wins when two sources disagree). Service Graph Connectors tag their data with a source identifier automatically.
📘
Core Concept — Why Not Use Raw Import Sets for CMDB? If you build a custom Import Set → Transform Map that writes directly to cmdb_ci_server, you bypass the IRE. That means no duplicate detection — you could create two CI records for the same physical server that came from different sources. No reconciliation — if Discovery and your CMDB import both have the same server with different OS versions, you don't know which wins. No relationship management — CI relationships are not automatically maintained. Service Graph Connectors solve all of this by routing through the IRE.

Service Graph Connector Architecture

Under the hood, a Service Graph Connector:

  1. Fetches data from the source system (via REST, file, JDBC, etc.)
  2. Loads it into an Import Set staging table
  3. Runs a Transform Map that produces a structured CMDB payload
  4. Submits the payload to the CMDB Identification and Reconciliation API
  5. The IRE processes the payload — identifies, reconciles, and writes to the appropriate CI table
⚠
Exam Trap — Service Graph Connector vs. Import Set Direct The exam will present a scenario: "A team imports server data from an Excel file using an Import Set and Transform Map directly to cmdb_ci_server. Why are duplicates appearing?" The answer is that they bypassed the IRE. The fix is to use a Service Graph Connector or route through the CMDB Identification API instead of direct table writes.
Section 4

Import Set API — Programmatic Import

The Import Set API (/api/now/import/{staging_table}) allows external systems to POST data directly into a ServiceNow staging table via REST, which then triggers the associated Transform Map automatically.

This is how external systems can push data into ServiceNow without needing to upload a file: the external system makes an HTTP POST request with the data as a JSON payload, ServiceNow receives it into the staging table, and the transform runs automatically.

MethodHow Data ArrivesTransform TriggerBest For
File upload (manual)User uploads CSV/ExcelManual or scheduledOne-time or manual loads
Scheduled Data SourceSN pulls from file share or URLScheduled jobRecurring batch loads
Import Set API (POST)External system pushes JSONAutomatic on receiveReal-time event-driven loads
Service Graph ConnectorConnector fetches from sourceScheduled or triggeredCMDB CI data
Section 5

Import Set States

Each row in an Import Set staging table has a state field that tracks what happened during the transform. These states appear in the import set run results and are critical for diagnosing import problems.

StateMeaning
InsertedA new record was created in the target table
UpdatedAn existing record was found and updated (via coalesce)
IgnoredRow matched an existing record but no fields changed — no update needed
ErrorThe transform failed for this row — check error message for details
SkippedRow was deliberately skipped by a transform script (onBefore returned IGNORE)
💡
Tip — Diagnosing Import Errors After a scheduled import, navigate to System Import Sets → Import Sets and open the most recent import set. Click the import set record to see the transform result counts (inserted/updated/ignored/error). Click into Error rows to see the specific error message. This is the fastest way to understand why data isn't landing correctly.
Section 6

MID Server Role in Imports

When ServiceNow needs to pull data from a source that is inside your corporate network (a database, a file share, an LDAP directory) and not publicly accessible from the internet, it uses a MID Server (Management Instrumentation and Discovery server).

The MID Server is a Java agent installed inside your network. It acts as a proxy: the ServiceNow instance tells the MID Server "go fetch data from this JDBC database using these credentials" and the MID Server does so from inside the network, then returns the data to the instance.

  • MID Servers are used for: JDBC data sources, LDAP imports, file share access, Discovery
  • They authenticate to internal systems using credentials stored in ServiceNow's Credential records
  • MID Servers are configured in MID Server → Servers
  • Data Sources that require a MID Server specify which MID Server (or cluster) to use
Exam Quick Reference

Common Exam Traps — Import Sets

  • Import Sets use a two-stage pipeline: staging table first, then Transform Map to target table
  • Import Set staging tables grow unboundedly — always configure cleanup/retention
  • Service Graph Connectors are the preferred method for CMDB CI data — they route through IRE
  • Direct Transform Map writes to cmdb_ci tables bypass the IRE — causes duplicates and reconciliation failures
  • Import Set API (POST /api/now/import/{table}) triggers transform automatically on receive
  • Import row states: Inserted / Updated / Ignored / Error / Skipped — Ignored means no change needed
  • MID Server required for JDBC, LDAP, or file shares inside the corporate network
Exam Checkpoint

Practice Questions

4 questions · Select an answer to see the explanation immediately.

Import Sets Quiz 1 / 4

What is the correct two-stage pipeline for the Import Set process?

💡 Explanation

B is correct. Import Sets use a two-stage pipeline: data loads into a staging table first, then a Transform Map processes and moves it to the target table. This staging approach allows data inspection and transformation before it reaches production tables. A skips the staging stage. C describes IRE, not Import Sets. D describes configuration, not the runtime pipeline.

A team uses a direct Transform Map to insert CI data into the cmdb_ci_linux_server table. What is the key risk of this approach?

💡 Explanation

B is correct. Direct Transform Map writes to CMDB CI tables bypass the IRE. Without IRE, identification rules don't run, so the same CI can be created multiple times as duplicates, and reconciliation between multiple sources breaks down. Service Graph Connectors should be used for CMDB CI ingestion instead. A, C, and D are not the primary concern.

An import row is processed and the result state is "Ignored". What does this mean?

💡 Explanation

B is correct. An "Ignored" state means the transform found a matching target record and compared values — and found no difference. No write was needed, so the row was ignored (not Inserted or Updated). This is the expected and correct behavior for unchanged records. A describes "Error". D describes a row where ignore = true is set, which shows as "Skipped" (not Ignored).

Which data source type almost always requires a MID Server to function?

💡 Explanation

B is correct. JDBC data sources connect to databases that are almost always inside the corporate network, unreachable from ServiceNow's cloud. A MID Server acts as the proxy. HTTPS REST endpoints accessible from the cloud and CSV uploads through the browser do not require a MID Server. The Import Set API (C) is a push model where the source calls ServiceNow.

Score: 0 / 4
Next: Transform Maps & Field Maps →