Running the Web Server Log Import DTS Task for a Large Site

An "out of memory" error might appear in the Data Warehouse event log for larger sites. In most cases, this is caused by the process running out of virtual memory (you can verify this with the Windows Performance Monitor application).

Standard 32-bit addresses can map a maximum of 4 gigabytes (GB) of memory. The standard address spaces of 32-bit Microsoft Windows 2000 processes are therefore limited to 4 GB. By default, 2 GB are reserved for the operating system, and 2 GB are made available to the application. If you specify a 3GB switch in the Boot.ini file of Windows NT Enterprise Edition or Microsoft Windows 2000 Advanced Server, the operating system reserves only 1 GB of address space, and the application can access up to 3 GB.

To avoid this error, enable the 3GB switch on the Data Warehouse computer before importing Web log files.

Enabling the 3GB switch requires two steps:

  1. Modify the Boot.ini file. For more information about the /3GB switch, see Windows 2000 Advanced Server Help.
  2. Modify the executable file that is running the log import DLL. Depending on how you import data into the Data Warehouse, you must modify the appropriate executable file.

The log import process is hosted by different executable files depending on how you launch log import.

To modify the executable running the Web log import DLL if you import data into the Data Warehouse by running DTS packages directly from the SQL Enterprise Manager User Interface

  1. At the system prompt, type: ImageCfg -l mmc.exe and then press ENTER.
  2. To verify that the 3GB switch is enabled, at the command prompt, type ImageCfgmmc.exe, and then press ENTER.

The current settings appear. If the 3GB switch is not present, you might need to enable it again, or rename the executable file and then try again.

To modify the executable running the Web log import DLL if you import data into the Data Warehouse with a DTS package scheduled to run with the SQL Enterprise Manager Scheduler

  1. At the system prompt, type: ImageCfg -l dtsrun.exe and then press ENTER.
  2. To verify that the 3GB switch is enabled, at the command prompt, type ImageCfg dtsrun.exe, and then press ENTER.

The current settings appear. If the 3GB switch is not present, you might need to enable it again, or rename the executable file and then try again.

To modify the executable running the Web log import DLL if you import data into the Data Warehouse with scripts and use Windows Script Host to perform your log imports

  1. At the system prompt, type: ImageCfg -l cscript.exe and then press ENTER.
  2. To verify that the 3GB switch is enabled, at the command prompt, type ImageCfg cscript.exe, and then press ENTER.

The current settings appear. If the 3GB switch is not present, you might need to enable it again, or rename the executable file and then try again.

See Also

Workflow for Running DTS Tasks

Best Practices for the Data Warehouse

ETL Process for the Web Server Log Import DTS Task

Running the Web Server Log Import DTS Task for a Large Site

Reports Based on Web Log File Data

Restarting, Ignoring or Deleting a Failed Web Server Log Import Task

Configuring the Web Server Log Import DTS Task

Commerce Server Performance Counters

Extending the Data Warehouse

Data Warehouse Schema

Troubleshooting the Data Warehouse Import Process

Copyright © 2005 Microsoft Corporation.
All rights reserved.