TechNet Radio: How Microsoft IT Leverages SQL Server 2008 SSIS Dataflow Engine Enhancements
About this Podcast
The Information Security Consolidated Event Management (ICE) system is a more than 30-terabyte data warehouse used by the Microsoft Information Security team to analyze network utilization events captured by various sources, including over 100 proxy servers, mail servers, Net logon servers, etc. The ICE database processes approximately 1 terabyte of log data each day and it has become a key component in the incident response process, in addition to forensics investigations. Analysis of the proxy data has empowered the Microsoft Information Security team to identify and remediate numerous security issues that would have gone undetected otherwise.
ICE version 4.0 is an ambitious project set to deliver almost real-time data and high-query performance to the security team using Microsoft SQL Server 2008. Moreover, ICE 4.0 is also designed to perform all sorts of data filtering and transformation during the data-loading process, so the schema of data stored in ICE is tailored for investigation analysis/reporting needs. An Online Analytical Processing (OLAP) cube is built on top of the ICE data warehouse to facilitate aggregated queries. Join this session to learn how enhancements to the Microsoft SQL Server Integration Service (SSIS) 2008 dataflow engine have significantly improved the performance of loading, filtering and transforming 1 terabyte of network log data into the ICE data warehouse.