Enhanced Performance and Caching for the Lookup Transformation
Performance enhancements to the Lookup transformation include faster cache loading and more efficient lookup operations. These enhancements are possible because of the following features:
-
The ability to take rows that do not have matching entries in the reference dataset and load those rows into the cache.
-
The ability to use separate data flows to load the reference dataset into the cache and to perform lookups on the reference dataset.
The Lookup transformation now includes the following caching options:
-
The reference dataset is a cache file (.caw), and is accessed by using a Cache connection manager.
-
The reference dataset is a connected data source in the data flow, and is accessed by using a Cache connection manager and a Cache Transform transformation.
-
The reference dataset is a table, view, or query that is fully or partially cached and accessed by using an OLE DB connection manager.
-
The cache can be shared between multiple Lookup transformations in a single package and between transformations in separate packages. For more information, see How to: Implement a Lookup Transformation in Full Cache Mode Using the Cache Connection Manager.
-
You can deploy a cache file together with a package. For more information, see How to: Create and Deploy a Cache for the Lookup Transformation.
For more information, see Lookup Transformation, Cache Connection Manager, and Cache Transform.
For a video that demonstrates how to configure the Lookup Transformation, see the video home page, How to: Implement a Lookup Transformation in Full Cache Mode (SQL Server Video), in the MSDN Library.
New ADO.NET Components
Integration Services now includes the following ADO.NET components:
-
An ADO NET source component that consumes data from a .NET Framework provider and makes the data available to the data flow. For more information, see ADO NET Source.
-
An ADO NET destination component that loads data into a variety of ADO.NET-compliant databases that use a database table or view. For more information, see ADO NET Destination.
New Data Profiling Task and Data Profile Viewer
The Data Profiling task is a new task in the Integration Services toolbox. You can use this task inside an Integration Services package to profile data that is stored in SQL Server. The information provided by the profile helps you identify potential problems with data quality. The Data Profiling task provides profiles that help identify data quality problems within individual columns and with column relationships:
- Profiles that help identify problems within individual columns
-
-
The distribution of lengths in the column values.
-
The percentage of null values.
-
The distribution of values in the column.
-
Column statistics for numeric columns.
-
Regular expressions that match string columns.
- Profiles that help identify problems with column relationships
-
-
Candidate key columns.
-
Functional dependencies between columns.
-
The inclusion of the set of values in one column in the set of values in another column.
For more information, see Data Profiling Task and Profiling Data with the Data Profiling Task and Viewer.
For a video that demonstrates how to use the Data Profiling task, see the video home page, How to: Use the Data Profiling Task (SQL Server Video), in the MSDN Library.
New Integration Services Connections Project Wizard
Use the Integration Services Connections Project Wizard to create a package that contains the connection information that you need to connect data sources and destinations. The wizard guides you through the steps of selecting data providers, configuring connection managers, and assigning connection managers to sources and destinations. For more information about when to use this wizard, see Creating a Package by Running a Wizard.
New Script Environment
Business Intelligence Development Studio now integrates seamlessly with Microsoft Visual Studio Tools for Applications (VSTA) environment. VSTA is the development environment in which a developer writes scripts for the Script task and the Script component.
VSTA supports both the Microsoft Visual Basic 2008 or Microsoft Visual C# 2008 programming languages. VSTA also enables you to add managed assemblies to a script at design time by browsing to the folder location. In addition, VSTA enables you to add a Web reference in your code that enables the code to use objects and methods provided by a Web service.
For SQL Server 2005 Integration Services (SSIS) packages that include Microsoft Visual Studio for Applications (VSA) scripts, VSTA converts those scripts.
Note: |
|---|
|
Breakpoints are not supported in the Script component.
|
For more information about VSTA in Integration Services, see the following topics:
Package Upgrade
You can upgrade your Integration Services packages from the format that Integration Services uses in SQL Server 2005 to the format that SQL Server 2008 uses. To upgrade your SQL Server 2005 packages, do one or more of the following procedures:
-
Use the dtexec command prompt utility (dtexec.exe) that comes with SQL Server 2008 to run the SQL Server 2005 package. When you use this method to run a SQL Server 2005 package, the upgrade is temporary, and the changes that result from the upgrade cannot be saved.
-
Add the SQL Server 2005 package to an existing project or open that package in SQL Server 2008 Integration Services. Integration Services will automatically upgrade the package. However, the upgrade is temporary. To permanently upgrade the package, you must save the package changes.
Note: |
|---|
|
To add a package to an existing package, on the Project menu, click Add Existing Package.
|
-
Create or open a SQL Server 2005 Integration Services project, and then use the SSIS Package Upgrade Wizard to upgrade all the packages in the project. This upgrade process is permanent.
For more information, see Upgrading Integration Services Packages.
Return to New Features and Enhancements by Area