Specify an Interval of Change Data
In the control flow of an Integration Services package that performs an incremental load of change data, the first task is to calculate the endpoints of the change interval. These endpoints are
datetime values and will be stored in package variables for use later in the package.
Before configuring the Execute SQL task to calculate the endpoints, the package variables that will store the endpoints have to be defined.
In SQL Server Data Tools (SSDT), open a new Integration Services project.
In the Variables window, create the following variables:
Create a variable with the
datetimedata type to hold the starting point for the interval.
This example uses the variable name, ExtractStartTime.
Create another variable with the
datetimedata type to hold the ending point for the interval.
This example uses the variable name, ExtractEndTime.
If you calculate the endpoints in a master package that executes multiple child packages, you can use Parent Package Variable configurations to pass the values of these variables to each child package. For more information, see Execute Package Task and Use the Values of Variables and Parameters in a Child Package.
After you set up the package variables for the interval endpoints, you can calculate the actual values for those endpoints and map those values to the corresponding package variables. Because those endpoints are
datetime values, you will have to use functions that can calculate or work with
datetime values. Both the Integration Services expression language and Transact-SQL have functions that work with
Functions in the Integration Services expression language that work with
Functions in Transact-SQL that work with
Date and Time Data Types and Functions (Transact-SQL).
Before you use any one of these
datetime functions to calculate the endpoints, you have to determine whether the interval is fixed and occurs on a regular schedule. Typically, you want to apply changes that have occurred in source tables to destination tables on a regular schedule. For example, you might want to apply those changes on an hourly, daily, or weekly basis.
After you understand whether your change interval is fixed or is more random, you can calculate the endpoints:
Calculating the starting date and time. You use the ending date and time from the previous load as the current starting date and time. If you use a fixed interval for incremental loads, you can calculate this value by using the
datetimefunctions of Transact-SQL or of the Integration Services expression language. Otherwise, you might have to persist the endpoints between executions, and use an Execute SQL task or a Script task to load the previous endpoint.
Calculating the ending date and time. If you use a fixed interval for incremental loads, calculate the current ending date and time as an offset from the starting date and time. Again, you can calculate this value by using the
datetimefunctions of Transact-SQL or of the Integration Services expression language.
In the following procedure, the change interval uses a fixed interval and assumes that the incremental load package is run daily without exception. Otherwise, change data for missed intervals would be lost. The starting point for the interval is midnight the day before yesterday, that is, between 24 and 48 hours ago. The ending point for the interval is midnight yesterday, that is, the previous night, between 0 and 24 hours ago.
On the Control Flow tab of SSIS Designer, add an Execute SQL Task to the package.
Open the Execute SQL Task Editor, and on the General page of the editor, select the following options:
For ResultSet, select Single row.
Configure a valid connection to the source database.
For SQLSourceType, select Direct input.
For SQLStatement, enter the following SQL statement:
SELECT DATEADD(dd,0, DATEDIFF(dd,0,GETDATE()-1)) AS ExtractStartTime, DATEADD(dd,0, DATEDIFF(dd,0,GETDATE())) AS ExtractEndTime
On the Result Set page of the Execute SQL Task Editor, map the ExtractStartTime result to the ExtractStartTime package variable, and map the ExtractEndTime result to the ExtractEndTime package variable.
When you use an expression to set the value of an Integration Services variable, the expression is evaluated every time that that the value of the variable is accessed.
After you calculate the starting point and ending point for a range of changes, the next step is to determine whether the change data is ready.
Next topic: Determine Whether the Change Data Is Ready