Data Sources & Scheduled Imports

Section 1

Data Source Types

A Data Source record (sys_data_source) defines the connection to an external system and how to retrieve data from it. Navigate to System Import Sets → Data Sources to view and create them.

Type	What It Does	MID Server Required?
File	Reads a CSV, Excel, XML, or JSON file. Can be uploaded manually, retrieved from a URL (HTTP/HTTPS), or pulled from an SFTP/FTP server.	Only if file is on internal network (SFTP/file share)
JDBC	Queries an external relational database directly using SQL. Requires a JDBC driver for the target DB type.	Yes — DB is usually inside the network
REST	Fetches JSON or XML from a REST API endpoint. Uses HTTP GET or POST with optional authentication.	Only if API is on internal network
LDAP	Reads directory data from an LDAP/Active Directory server. Used for user imports.	Yes — AD is always internal
Custom (Script)	Runs a JavaScript script to populate the staging table. Full flexibility for unusual sources.	Depends on script implementation

⚠

Exam Trap — JDBC Always Needs MID Server JDBC Data Sources almost always require a MID Server because enterprise databases are inside the corporate network and not accessible from the internet (where the ServiceNow instance runs). If a JDBC import is failing and the question asks why, check whether the MID Server is running and whether it has the correct JDBC driver installed.

Section 2

Scheduled Import Jobs

A Scheduled Import Set combines a Data Source with a schedule — it automatically pulls data from the source, loads it into the staging table, and runs the Transform Map on a recurring basis. This is how production imports run: nightly user syncs from Active Directory, hourly CI updates from discovery tools, daily asset feeds from an ITAM system.

To create one, navigate to System Import Sets → Scheduled Import Sets → New. Key configuration fields:

Name — descriptive label
Data source — which Data Source record to use
Run as — which user account to run as (determines ACLs applied)
Run — frequency: Daily, Weekly, Periodically, On Demand
Time — when to run (for Daily schedule)
Execute now button — runs the import immediately (useful for testing)

📘

Core Concept — Scheduled Import vs. Scheduled Job A Scheduled Import Set is a specialized wrapper — it handles the load + transform pipeline. A regular Scheduled Job (sys_trigger) is a generic timer that runs arbitrary JavaScript. They are different record types. Import Sets have their own specific record type and run history tracking. Don't confuse them on the exam.

Section 3

Import Set Run History and Diagnostics

Every time a Scheduled Import Set runs, ServiceNow creates a run history record. To view it: navigate to System Import Sets → Import Sets. Each record represents one run and shows:

Number — import set identifier
State — Loaded, Running, Completed, or Error
Created — when the import set was created (i.e., when data was pulled)
Transform results — count of Inserted / Updated / Ignored / Error rows

Clicking into an import set record shows the individual import set rows — one per row of source data — each with its own state and error message if applicable.

Common Failure Scenarios

Symptom	Likely Cause	Fix
Import set stuck in "Loading" state	MID Server down or unreachable	Check MID Server status; restart if needed
All rows show Error state	Transform Map misconfiguration or target field mandatory but empty	Check error message on first error row; fix Transform Map
Duplicates appearing in target table	Coalesce not configured or coalesce field values not matching	Add/fix coalesce field on Field Map; verify unique identifier in source data
Import runs but target table unchanged	All rows state is "Ignored" (no changes) or Transform Map not associated	Verify Transform Map is linked to the Data Source/Import Set
Data missing after import	onBefore script setting `ignore = true` for too many rows	Review onBefore script logic; add logging to diagnose skipped rows

Section 4

Data Source Authentication

Data Sources that connect to secure systems need credentials. ServiceNow stores credentials securely in Discovery Credentials records (or Connection Records for IntegrationHub). The Data Source references the credential record by name — the actual username and password are stored encrypted, not exposed in the Data Source configuration.

Basic auth — username + password for HTTP/REST endpoints or file share access
OAuth 2.0 — token-based auth for modern REST APIs; tokens auto-refresh
JDBC auth — database username + password; MID Server uses these credentials
SSH key — for SFTP sources; key stored in credential record

🚫

Warning — Never Hard-Code Credentials Do not store usernames or passwords in Data Source configuration fields or Transform Map scripts. If credentials change, you would need to find and update every script. Always use credential records and reference them by name. Also, never log credentials in Transform Map scripts — import logs are visible to many users.

Section 5

Incremental vs. Full Imports

When running scheduled imports, you must decide whether to load all records from the source every time (full load) or only records that have changed since the last import (incremental/delta load).

Strategy	How It Works	Pros	Cons
Full Load	Pull all records every run	Simple; ensures complete sync	Slow on large datasets; creates large staging tables
Incremental (Delta)	Pull only records changed since last run (using a "last modified" timestamp filter)	Fast; lightweight staging table	Source must support timestamp filtering; deleted records not captured
Event-driven (Import Set API)	Source pushes changes when they happen	Real-time; minimal latency	Source must support webhooks or push; more complex to implement

⚠

Exam Trap — Incremental Imports Miss Deletes Incremental imports using a "last modified" filter will never capture deleted records in the source system. If a user is deactivated in HR and you only import changes, that deactivation may never reach ServiceNow. Full loads (where coalesce handles updates) catch this only if you also add logic to deactivate records no longer in the source. The exam tests this gap.

Exam Quick Reference

Common Exam Traps — Data Sources & Scheduled Imports

JDBC Data Sources almost always require a MID Server (databases are inside the network)
Scheduled Import Set = Data Source + Transform Map + schedule — a single automated pipeline
Import set states: Loaded → Running → Completed. Row states: Inserted / Updated / Ignored / Error / Skipped
Credentials should always be stored in Discovery Credential records — never hard-coded in scripts
Incremental imports miss deleted records — use full loads with deactivation logic to handle removes
"Execute now" button on Scheduled Import Set triggers an immediate run without waiting for schedule

Exam Checkpoint

Practice Questions

4 questions · Select an answer to see the explanation immediately.

Data Sources Quiz 1 / 4

A Scheduled Import Set needs to pull data from an internal Oracle database. What infrastructure is almost certainly required?

💡 Explanation

B is correct. JDBC data sources (like Oracle databases) live inside the corporate network behind firewalls, unreachable from ServiceNow's cloud. A MID Server deployed inside the network acts as an outbound proxy, connecting to the database and relaying results to ServiceNow. A and C are not standard solutions for JDBC. D is impossible — databases do not expose REST APIs natively.