Service Graph Connectors & the MID Server
ServiceNow cannot always reach your infrastructure directly. Service Graph Connectors (SGC) are pre-built integrations that pull discovery data from third-party sources — cloud providers, CMDBs, monitoring tools — and route it through ServiceNow's Identity and Reconciliation Engine (IRE) to create accurate, deduplicated CI records. This topic introduces the MID Server, the on-premise bridge that makes it all work.
Why Direct Table Writes Break Your CMDB
In the previous topics you learned how Import Sets work: data lands in a staging table, Transform Maps move it to the target CMDB table, and Coalesce fields decide whether to create a new record or update an existing one. That pipeline is powerful for one-time migrations and scheduled batch loads.
But imagine you have five different tools that all know about your servers — AWS Config, VMware vCenter, a monitoring agent, a manual spreadsheet import, and your own homegrown CMDB. All five start writing to the same cmdb_ci_server table. What happens?
- Duplicates. Tool A writes a server with IP 10.0.0.5. Tool B also writes it, but with a slightly different hostname. Now you have two CI records for the same physical machine.
- Overwriting good data with bad data. Your authoritative source set serial_number = SN-12345. A less-reliable source overwrites it with serial_number = UNKNOWN. The good data is gone.
- No audit trail. You cannot tell which source populated which field, making it impossible to debug quality problems.
Direct Table API writes (a REST POST directly to cmdb_ci_server.do?sysparm_action=insert) bypass IRE entirely. This is the single most important exam trap in this domain: direct writes skip IRE, which means no deduplication, no source precedence rules, no reconciliation. Service Graph Connectors exist precisely to fix this — they route all incoming data through IRE before anything touches the CMDB.
cmdb_ci_server (Table API) bypasses IRE completely.
No deduplication, no reconciliation, no source precedence. This is always wrong for
multi-source CMDB environments. SGC data, by contrast, always flows through IRE.
This distinction appears in scenario questions: "A developer writes a script that inserts
CIs via Table API. What quality risk does this introduce?"
What Is a Service Graph Connector?
A Service Graph Connector (SGC) is a pre-built integration package available on the ServiceNow Store that connects a specific third-party data source to your ServiceNow CMDB. Each SGC knows:
- How to authenticate to the source system (API key, OAuth, certificate)
- What data to pull and how often (scheduled polling vs. push/webhook)
- How to map the source's data model to ServiceNow CI classes
- How to route that data through IRE rather than writing directly to tables
Instead of a developer spending weeks building a custom integration for AWS, you install the "Service Graph Connector for AWS" from the Store, configure credentials, and it starts pulling EC2 instances, RDS databases, S3 buckets, and VPCs as properly classified CIs — all deduplicated through IRE.
Available Service Graph Connectors (Examples)
The MID Server — ServiceNow's On-Premise Proxy
Many of your data sources sit inside a private network — on-premise servers, private cloud VPCs, internal databases. ServiceNow runs in the cloud (on ServiceNow's own data centers). These two worlds cannot talk directly because your firewall blocks inbound connections from the internet.
The Management, Instrumentation, and Discovery (MID) Server solves this problem. It is a Java application that you install on a server inside your private network. The MID Server makes an outbound connection to your ServiceNow instance — outbound connections are allowed by most firewalls — and maintains a persistent, bidirectional channel. ServiceNow can then ask the MID Server to reach internal resources on its behalf.
How the MID Server Communication Works
ecc_queue table inside the instance. The MID Server is not contacted directly — ServiceNow simply writes a record and waits.Key MID Server Facts for the Exam
- Operating System: Windows or Linux. Java 11+ required.
- Network requirement: Outbound HTTPS (port 443) to your ServiceNow instance. No inbound ports needed.
- Authentication: MID Server authenticates to ServiceNow using a dedicated service account with the mid_server role.
- ECC Queue: The External Communication Channel queue is the message bus between ServiceNow and MID Server. Input = commands to MID. Output = results from MID.
- High availability: Multiple MID Servers can exist. They are assigned to IP ranges or applications. Cluster mode provides redundancy.
- Not always required: For cloud APIs accessible from ServiceNow's network (public AWS endpoints), SGC can call the API directly without a MID Server. MID Server is required when the target is on a private network.
SGC Data Flow — End to End
Understanding the exact path data takes from a third-party source to a CI record in your CMDB is critical for the exam. Let's trace it step by step.
Step-by-Step: SGC Pulls AWS EC2 Instances
- Scheduled trigger fires. The SGC has a scheduled job configured to poll every 4 hours. At 9:00 AM the job fires.
-
SGC calls AWS EC2 API. Using stored credentials (an IAM role or
access key), the SGC calls
DescribeInstances. This returns raw JSON with instance IDs, AMIs, tags, IPs, etc. If a MID Server is configured, this API call is proxied through the MID Server. - Payload transformation. The SGC transforms the raw AWS JSON into ServiceNow's internal Identification and Reconciliation API (IRE API) format. At this stage, data is NOT written to any CMDB table. It is packaged as a structured payload.
- IRE receives the payload. The payload is handed to IRE. IRE looks at each CI in the payload and asks: "Does this CI already exist in the CMDB?" using Identification Rules (covered deeply in Part 4). If a match is found, this is an update. If not, this is a new CI.
-
Reconciliation decides field values. For update scenarios, IRE checks
Reconciliation Rules: which data source has authority for each field? If AWS says
cpu_count=4but a prior Discovery scan saidcpu_count=8, Reconciliation Rules determine which wins. - CMDB record created or updated. IRE writes the final, reconciled values to the appropriate CMDB table (e.g., cmdb_ci_vm_instance for an EC2 instance). A last_discovered timestamp is updated. The discovery_source field is set to the SGC name.
- IRE log updated. The result of each CI's processing is written to the IRE log, not the Import Set log. This is a key distinction — SGC processing is visible in IRE logs, not Transform Map logs.
SGC vs. Direct Table API vs. Import Sets — Full Comparison
Knowing when to use each ingestion method — and what each method guarantees — is a recurring exam theme. Study this table carefully.
| Dimension | Service Graph Connector | Import Set + Transform Map | Direct Table API (REST) |
|---|---|---|---|
| Goes through IRE? | ✅ Yes — always | ⚠️ Only if Transform Map calls IRE API (requires explicit configuration) | ❌ No — bypasses IRE entirely |
| Deduplication | ✅ Automatic via IRE | ⚠️ Only via Coalesce field in Transform Map | ❌ None — can create duplicates |
| Multi-source reconciliation | ✅ Full source precedence via Reconciliation Rules | ❌ Last-write-wins (no source weighting) | ❌ None |
| Staging table used | ❌ No staging table | ✅ Yes — sys_import_set |
❌ No |
| Best for | Continuous sync from cloud/ITSM/monitoring tools | One-time migrations, periodic batch loads, CSV/Excel files | Quick scripted updates by developers who accept the risks |
| Error visibility | IRE logs + connector job logs | Import Set row status + Transform log | HTTP response codes only |
| Requires MID Server? | Only for on-premise targets | Only if data source is on-premise | No |
| Custom CI class mapping | Preconfigured by ServiceNow per connector | Fully customizable in Transform Map | You specify the table in the URL |
Understanding SGC Categories
SGCs are organized into categories based on what type of data source they connect to. Knowing the categories helps you answer "which connector should be used for X" questions.
Cloud Infrastructure Connectors
These connectors pull compute, storage, networking, and managed service resources from cloud providers. They are the most common SGC category.
- AWS: EC2, RDS, EKS, Lambda, S3, VPC, CloudFormation stacks, IAM roles
- Azure: Virtual Machines, App Services, AKS, SQL Managed Instance, Blob Storage
- GCP: Compute Engine VMs, GKE clusters, Cloud SQL, Cloud Storage
Cloud connectors typically use the cloud provider's read-only IAM permissions. They do NOT modify any resources in the cloud — they only read.
Virtualization & On-Premise Connectors
These always require a MID Server because the data source is on-premise.
- VMware vCenter: VMs, hosts, datastores, clusters, resource pools
- Microsoft Hyper-V: VMs, virtual switches
- Nutanix: VMs, clusters, storage
Software Asset & DevOps Connectors
Track software, repositories, and pipelines as CIs in the CMDB.
- GitHub / GitLab: Repositories, organizations, branches, pipelines
- Jira: Projects mapped to application CIs
- Dynatrace / Datadog: Monitored services and hosts as CIs
Security & Vulnerability Connectors
- Qualys: Maps scan results to existing CIs, enriching vulnerability data
- Tenable.io: Asset inventory with CVE associations
- CrowdStrike: Endpoint agents mapped to CIs
Configuring a Service Graph Connector
All SGC configurations follow the same general workflow. Understanding this helps you answer procedural questions on the exam.
Installation
- Search the ServiceNow Store for the connector (e.g., "Service Graph Connector for AWS").
- Install the application to your instance. This creates the connector's scheduled jobs, data source records, and IRE configuration.
- The connector appears under Configuration → Connections & Credentials or under its own application menu.
Authentication Setup
Each connector has a Connection & Credential record. You configure:
- Connection alias — A named connection (e.g., "AWS Production Account"). Allows multiple accounts per connector.
- Credential record — API key, OAuth client ID/secret, certificate. Stored encrypted using ServiceNow's credential vault.
- MID Server selection — If required, which MID Server to route through.
Data Scope Configuration
Most connectors let you scope what gets imported:
- Filters/tags: Only pull AWS resources tagged with
Environment=Production - Regions/accounts: Pull from us-east-1 and eu-west-1 only
- CI classes: Only pull EC2 instances, skip Lambda functions
Scheduling
Each SGC creates one or more Scheduled Data Imports. You can configure:
- Frequency (every 4 hours, daily, weekly)
- Full sync vs. incremental sync (delta-based)
- Run immediately for an initial baseline population
How SGC Works with IRE (Preview)
You will study IRE in depth in Part 4. For now, you need to know enough to understand what happens when SGC data arrives at IRE. This section introduces the concepts; Part 4 covers the full detail.
Three Questions IRE Answers for Every CI
-
"Does this CI already exist?" (Identification)
IRE uses Identification Rules to match incoming CI data against existing CMDB records. For a server, this might check: serial number, MAC address, or hostname. If a match is found, this is an update. If not, this is a new CI. -
"If it exists and I have multiple sources — what is the true value?" (Reconciliation)
IRE uses Reconciliation Rules to determine source precedence per field. "AWS is authoritative for instance_type. Discovery is authoritative for cpu_count. Manual entry is authoritative for owned_by." -
"Are there duplicate CIs for this same physical/logical entity?" (Deduplication)
IRE detects duplicate CI records and can either merge them automatically or flag them for manual review via Remediation Playbooks.
The discovery_source Field
When IRE processes SGC data, it sets the discovery_source field on the CI record to the name of the SGC. This is how you can later query "which CIs came from AWS SGC?" You can filter cmdb_ci where discovery_source = ServiceNow AWS.
What to Check When SGC Is Not Creating CIs
- Check the Scheduled Import job log — did the job run? Did it pull data?
- Check the CMDB IRE Log (
cmdb_ire_event) — did IRE receive the data? Were there identification errors? - Check Identification Rules — is there a rule for the CI class being imported? (Part 4)
- Check MID Server status — if on-premise, is the MID Server up and connected?
- Check credentials — have API keys expired?
cmdb_ci_vm_instance class. If IRE cannot identify whether incoming records
are new or existing (because no rule exists), it may drop or fail them rather than
creating new records.
MID Server Roles, Clusters, and Management
Because MID Server is introduced here and will appear repeatedly in Discovery (Part 3), let's cover the management concepts the exam tests.
MID Server Applications
A single MID Server can handle multiple types of work. Each type is called an Application:
- Discovery — Runs probes and sensors to scan IP ranges (Part 3)
- Service Graph Connectors — Proxies API calls to on-premise sources
- Orchestration — Runs workflow activities on remote systems
- Event Management — Receives alerts from monitoring tools
- IntegrationHub — Executes spoke actions against on-premise systems
MID Server Clusters
For high availability and load distribution, you can group MID Servers into a cluster. Work items are distributed across cluster members. If one MID Server fails, others pick up its work. Key exam points:
- Cluster members must be in the same network segment to be useful for Discovery
- Primary and secondary MID Server designation within a cluster
- Cluster affinity — certain work items can be pinned to specific MID Servers
MID Server Validation
Before a MID Server can be used, it must be validated. Validation confirms the MID Server meets security and version requirements set by your ServiceNow instance. Key facts:
- MID Server auto-upgrade is configurable — ServiceNow can push updates automatically
- If a MID Server is not validated, jobs assigned to it will not run
- Validation status is visible at MID Server → Servers
MID Server IP Ranges
For Discovery, each MID Server is assigned one or more IP ranges. When a Discovery schedule fires for a range, ServiceNow routes the work to the MID Server that owns that range. This ensures that a MID Server in the US east data center handles US east server scans, not a MID Server in Europe.
| MID Server Property | Where to Configure | Why It Matters |
|---|---|---|
| Service account credentials | config.xml on the MID Server host |
Authenticates MID Server to ServiceNow instance |
| IP Range assignment | MID Server record → IP Ranges related list | Routes Discovery work to correct MID Server |
| Application assignment | MID Server record → Applications related list | Controls what types of work this MID Server handles |
| Cluster membership | MID Server record → Cluster field | High availability and load distribution |
| Validation status | MID Server record → Validated field | Must be Validated before use; auto-checked on startup |
SGC & MID Server — Key Facts
- SGC always routes through IRE. Direct Table API writes bypass IRE — this is always wrong for multi-source environments.
- SGC does NOT use Import Sets. Look for SGC activity in IRE logs and connector job logs, not Transform Map logs.
- MID Server = outbound proxy inside your private network. Makes outbound connections to ServiceNow — no inbound firewall holes needed.
- MID Server required for on-premise targets (vCenter, Hyper-V, internal DBs). Optional for cloud APIs accessible from ServiceNow's network.
- ECC Queue = message bus between ServiceNow and MID Server. Input queue = instructions to MID. Output queue = results from MID.
- MID Server must be Validated before it can run jobs. Validation auto-runs on startup.
- discovery_source field = which SGC or Discovery schedule created/last updated the CI.
- Full sync first, then incremental. Full sync establishes baseline; incremental only pulls deltas.
- SGC troubleshooting order: Job log → IRE log → Identification Rules → MID Server status → Credentials.
- Multiple MID Servers can form a cluster for high availability and IP range-based routing.
Term Grid — SGC Key Vocabulary
Practice Questions
Click any question to reveal the answer and explanation.
cmdb_ci_server. What is the primary CMDB quality risk of this approach?Practice Questions
4 questions · Select an answer to see the explanation immediately.
A Service Graph Connector (SGC) is configured to ingest data from a cloud provider. How does SGC process the ingested CI data before writing it to CMDB?
B is correct. SGC always routes through the IRE. This is the key distinction that makes SGC the preferred CMDB ingestion method — it ensures identification rules run, deduplication occurs, and reconciliation rules determine field authority. A is wrong (that's the dangerous direct-write pattern). C is wrong (SGC does NOT use Import Sets). D is for agentless Discovery probes.