Troubleshoot a SharePoint farm by using SPDiag

Applies To: Office SharePoint Server 2007

This Office product will reach end of support on October 10, 2017. To stay supported, you will need to upgrade. For more information, see , Resources to help you upgrade your Office 2007 servers and clients.

Topic Last Modified: 2016-11-14

SPDiag is a tool used to collect, filter, and display data from a SharePoint farm for troubleshooting purposes. SPDiag is a read-only tool, and cannot make any changes to a farm. You can use SPDiag to help you to identify problems yourself, or as a way to collect the data needed by support personnel to help troubleshoot your farm.

Some information in this article assumes that you are using SPDiag version 2, included in SharePoint Administration Toolkit version 4. For more information, see What's new in SPDiag version 2.

The information in this section will help you understand how to create and import projects; filter and collect data; generate graphs and reports; and export data to a file.

Security Note:

The user running SPDiag must have Windows server administrator privileges on all servers in the farm, as well as SharePoint farm administrator privileges and db_owner privileges on the SPDiag project database. We recommend using an existing administrator account, such as the farm administrator account, rather than creating a new one. Running SPDiag with insufficient permissions might result in incomplete data collection, as well as connectivity issues or slow performance.

Using SPDiag to troubleshoot a farm involves the following general steps:

1. Creating or importing a new project.

2. Ensuring that log files and performance counter files are available for each farm server, and note log file locations for each farm server.

3. Selecting a date/time range for the data you want to analyze.

4. Selecting the performance counters and log files for analysis.

5. Collecting data from the farm servers.

6. Displaying data in performance counter graphs and custom reports.

7. Optionally exporting data to be analyzed by consultants or support personnel.

8. Optionally using the Diagnostics tab to execute preconfigured rules that check for common issues

   For more information about using the Diagnostics tab, see SPDiag Diagnostics tab.

In this section:

• Creating a new project

• Collecting and importing data using SPDiag from the command line

• Working with data

Creating a new project

An SPDiag project consists of a collection of data representing up to seven consecutive days of IIS, ULS and event logs, and performance counter log data. A project is stored in a SQL Server database on a database server that you specify. A project can be saved indefinitely, and data in the project can be reused many times to create snapshots and reports based on different sets of data stored in the project database.

Starting in SPDiag version 2, you can use one of the following two File menu commands to create a project:

• New Project, which is used to create a blank project database. Then, you use the SPDiag user interface to specify what data you want to collect from the connected target farm.

• Import Project, which allows you to import data from a remote farm that was collected using SPDiag in command line mode. For more information about importing a project, see Collecting and importing data using SPDiag from the command line later in this topic.

To create a new project in SPDiag, use the following procedure.

Create a new project

1. On the File menu, click New Project.

2. In the New Project dialog box, in the Database Server field, type the name of the database server on which you want to store the project in the format <servername\database instance>.

3. In the Project Name field, type a name for the project. The name will be used as the database name. The project name can only contain alphanumeric characters and underscores.

4. In the Project Description field, you can optionally enter descriptive text about the project.

5. Click Create to create the project database.

   A message displays the following text: "Creating project, this may take a few minutes."

6. Click OK.

Collecting and importing data using SPDiag from the command line

In SPDiag version 2, you can use the command line to collect data from all servers in a farm and then import the data into a new project on another computer running SPDiag. This is useful if you do not want to install the SharePoint Administration Toolkit on a server in the target farm; if you want to collect data to be analyzed on a remote computer or by a third party; or if you want to automatically capture log data as a triggered event.

You can run SPDiag from the command line on a computer where it has been installed as part of the SharePoint Administration Toolkit, or where the required files have been manually copied. For a list of the files you need in order to run SPDiag without installing the SharePoint Administration Toolkit, see "Required files" later in this section.

SPDiag can be run using the following parameters:

spdiag.exe -help [<mode>]

Executing spdiag.exe with the -help parameter displays the available parameters and usage examples. You can also use -help <mode> using one of the two modes described in the table below for help specific to that mode.

spdiag.exe -mode <mode> [<parameters>]

The -mode parameter is used to collect data. Several additional parameters are available for specifying the target folder for storing collected data, the time range for data collection, and secondary data source locations.

There are two data collection modes available from the command line, as shown in the following table. Only one mode parameter can be used at a time.

To collect data using SPDiag from the command line, use the following procedure.

Collect data using SPDiag from the command line

1. If the required files do not exist on the target server, either install SPDiag from the SharePoint Administration Toolkit or manually copy the required files to the server.

   For a list of the files you need, see "Required files" later in this section.

2. At the command prompt, change the directory to the folder that contains spdiag.exe.

3. To collect snapshot data, type spdiag.exe -mode snapshot -targetfolder <path>.

4. To collect log data, type spdiag.exe -mode logs -targetfolder <path> -start <"mm/dd/yyyy hh:mm"> -end <"mm/dd/yyyy hh:mm">. You can optionally add any or all of the additional<log type>logpath parameters if there are additional locations on the target farm servers from which you would like to collect data.

5. When the commands have completed, copy the specified target folder to the desired network location, portable media or removable device. Depending on the volume of the data, you may want to add the folder contents to a compressed archive, such as a .zip file.

Importing collected data into a new project for offline mode analysis

You can import data collected using the command line into a new SPDiag project for analysis in offline mode. You can also import updated data into a project later to supplement existing data. When you use SPDiag in offline mode, most SPDiag functionality is available, but analysis is limited to the existing data.

Import data into a new project

1. Open the SPDiag tool.

2. On the File menu, click Import Project.

3. In the Import Project dialog box, in the Database Server field, type the name of the database server on which you want to store the project in the format <servername\database instance>.

4. In the Project Name field, type a name for the project. The name will be used as the database name. The project name can only contain alphanumeric characters and underscores.

5. In the Project Description field, you can optionally enter descriptive text about the project.

6. In the Log root folder field, click the Browse button to browse to the location where the collected log files are stored.

7. In the From field, specify the start time for data to import into the project.

8. In the To field, specify the end time for data to import into the project.

9. In the Log upload speed field, select a value from the drop-down menu. Available values are Low, Medium, and High, and correspond with the log upload speed settings in the Installing SPDiag section of Set up and configure SPDiag.

10. Click Import to create the project database.

    A message displays the following text: "Importing project, this may take a few minutes."

11. Click OK.

Required files

The following files in the folder C:\Program Files\Microsoft\SPAdministrationToolkit\SharePoint Diagnostics are required for SPDiag to be executed from the command line:

-
Microsoft.Office.SharePoint.SPDiag.AnalysisRules.dll

-
Microsoft.Office.SharePoint.SPDiag.dll

-
Microsoft.Office.SharePoint.SPDiag.MSChartWrapper.dll

-
Microsoft.Office.SharePoint.SPDiag.SharePointWrapper.dll

-
PartitionedViewSchema.sql

-
SPDiag.exe

-
SPDiag.exe.config

-
Spdiag.ico

If you want to run SPDiag from the command line without installing the SharePoint Administration Toolkit, you need to manually copy the required files to a single folder on the target computer. You can copy the files from a computer where SPDiag has been installed and add them to an archive, such as a .zip file, for ease of use.

Working with data

SPDiag collects several different kinds of data, and amalgamates collected data in displays and reports. There are three primary views in SPDiag version 2: Snapshot, Trends, and Diagnostics.

The Snapshot view is displayed by default when you open SPDiag. Snapshot data, which contains information about the logical structure of the farm as well as details about the hardware and software in the farm, is collected by updating the Snapshot pane. Snapshot data is static once it has been collected by SPDiag, and is not used in reports or graphs in the Trends pane.

In Trends view, SPDiag provides filters that allow you to select specific data points from log files and performance counter output data that has been captured on servers in the farm. This data is useful when you want to correlate data from a specific time range in the past with observed performance issues.

You can also use Live Capture to collect data in real time by creating and running a data collector set on a target server in the farm. This is useful when you want to observe the results of real time events on farm performance. For example, you could start a Live Capture session and then manually start a backup job on the farm to observe the results. See the "Collecting performance counter data using Live Capture" section later in this article for more information.

SPDiag uses a SQL Server database as the repository for collected data. Each logical project is stored in a single database that can contain up to seven days of data. SPDiag can collect performance data from IIS logs and performance counters on the farm servers, and can also collect live data from the servers using data collector sets you configure in Live Capture.

All captured performance counters, logs and report data points are retrieved from the project database as needed. Once you select a specific data point to be displayed for a certain timeframe, that data is stored in the project database and will not be collected from the farm servers again.

If you want to discard stored data, you can purge the project database by clicking Purge Data on the Data menu. By selecting the date range for data you want to purge, all data for the date range will be removed from the project database. Data that is stored on the farm servers, such as .blg files created by data collector sets, will not be removed from the farm servers when you purge data in SPDiag.

Read the sections below for details on how to filter, collect, display and export data.

Security Note:

The user running SPDiag must have Windows server administrator privileges on all servers in the farm, SharePoint farm administrator privileges and db_owner privileges on the SPDiag project database. We recommend using an existing administrator account, such as the farm administrator account, rather than creating a new one. Running SPDiag with insufficient permissions might result in incomplete data collection, as well as connectivity issues or slow performance.

Snapshot view

SPDiag collects information about the hardware, software and logical structure of the connected farm and displays it in the Snapshot pane.

You can export the snapshot to an XML file that can be saved as a record of your farm configuration and topology, and can supplement other troubleshooting documentation and maintenance records.

To update the information in the Snapshot pane, right-click on an object in the tree view pane and then click Update. All objects under the selected object will then be updated to reflect their current values.

Selecting and collecting data in Trends view

Once you have collected data from the farm, the data points that have been collected are rendered in their respective panes on the Trends tab. You can also select data from the collected log files and performance counters for inclusion in a custom report in the Custom Reports pane. You can also display collected performance counter data in the Performance Monitor pane.

The Consolidated Logs View pane displays the contents of all collected IIS, ULS and event log files by object. You can use the bottom scroll bar to read the raw log entries in this pane.

The Performance Monitor pane displays performance counter data that you select in a graphical format. You select the data to display by setting a filter that extracts data from the collected performance counter files.

The Custom Reports pane allows you to view reports that list the top sites, slow requests, and failed requests from the collected data. You can filter this data in a variety of useful ways.

Data selection and collection for trend analysis is done in the Trends pane, which you access by clicking the Trends tab in the main SPDiag window.

The master filter bar at the top of the Trends pane governs the time frame and scope (the Web servers and Web applications) of the data that is displayed in all three sections of the Trends view. You use the fields in the master filter bar to specify the time frame and scope for collecting data, and for displaying data that has already been uploaded into the project database. After you make changes to the time frame or scope, click the Refresh button to update the data displayed in the Trends view.

First, on the master filter bar, select the time frame, the servers, and the Web applications from which you want to collect data.

Select time frame and collection scope

1. In SPDiag, click the Trends tab.

2. In the Analyzing calendar control, select the date from which to collect data.

   Tip

   The first time you select a date in an SPDiag project, you can choose any date in the past. This sets the project’s date range of seven consecutive days, beginning at the first date you chose. Once this date range has been set, any date outside the date range will be grayed out in the Analyzing calendar control. If you want to analyze data from another time period, you must create a new SPDiag project.

3. In the From control, select the local server time at which to start collecting data.

4. In the To control, select the local server time at which to stop collecting data.

5. Click the SharePoint Scope button to select the servers and Web applications from which to collect data.

   In the Select servers and Web applications dialog box, all servers and Web applications in the farm are listed. Select All Servers and Select All Web Applications are selected by default. Use the check boxes next to each server and Web application to select the ones from which you want to collect data.

Next, select log file data you want to collect for the time frame and scope you specified in the last procedure.

Select and collect log file data

1. In the Consolidated log view pane, click the Filter Logs button.

2. In the Merged Logs Filter dialog box, you can select IIS logs, ULS logs, and event logs. When you select any of the three options, SPDiag will collect those logs inclusively for the time frame and scope you specified.

3. You can exclude records from collection by defining a filter under each log type. In the Hide entries based on this filter section under a given log type, use the And/Or field to specify whether a filter statement is in addition to or in exclusion of the prior filter statements, the Field name field to select the record you want to add to the filter, the Operator field to set statement operators (such as greater than, equals, or contains), and use the Value field to specify the value to use in the filter statement.

   It is important to remember that the merged logs filter is exclusionary, and that any records you select in a filter will be excluded from data collection.

   Note

   You can use filters to explicitly display selected data by using negative operators. For example, if you have selected IIS to collect IIS logs, you can collect only records specific to the client IP address 192.168.0.32 by setting the following filter:

   • Field name: Client IP address

   • Operator: Not Equal

   • Value: 192.168.0.32

   The use of the Not Equal operator will exclude from collection all records in which the client IP address does not equal the specified value. You can add other OR statements with negative operators to the filter to explicitly view other records.

4. Click Ok to collect the data you have selected and close the Merged Logs Filter dialog box. You can also click Apply if you want to collect the log file data and leave the filter dialog box open.

   Note

   If you click Apply to collect the selected data and then click Ok, the data will be collected from the farm servers again. If you want to close the Merged Logs Filter dialog box without refreshing the data, click Cancel.

Next, select performance counter data for the time frame and scope you specified.

Select and collect performance counter data

1. In the Performance Monitor pane, click the Filter Counter button.

2. In the Performance Counter dialog box, select a value from the Servers menu. You can select a server name from the menu to display specific performance counters from that farm server. You can also select Farm, which displays calculated farm-wide measurements based on IIS log data.

   1. If you selected a server from the Servers menu, a list of expandable performance counter categories will appear in the field below. Expand a category and select a performance counter from the list of available queries by clicking it.

   2. If you selected Farm from the Servers menu, an expandable tree entitled SharePointRequests will appear in the field below. Expand the parent item and select a query from the list of available queries by clicking it.

      

3. Selecting a query will open its available instances in the Instances of selected object pane on the right side of the dialog box. Click the instance you want to select.

   1. If you selected a server from the Servers menu, selecting a performance counter will display its available instances, which might be different for each counter. If no instances appear in the Instances of selected object pane, the main performance counter object will be used.

   2. If you selected Farm from the Servers menu, the instances _Total and _Master-Filter will be displayed for each query you select. Selecting the _Total instance will collect this data from all farm servers, and _Master-Filter will collect data only from the servers that are currently enabled in the SharePoint Scope on the master filter bar.

4. You can also select a color for the displayed data, and the scale for the display, and then click Add. The selected performance counter instance will appear in the Performance Monitor pane.

   Repeat this step for each performance monitor counter you want to add to the display.

5. When you have selected and added all the performance counter instances you want to collect, click OK to collect the data from the farm.

You can modify the performance counter display by hiding counters. In the list of displayed counters, clear the check box in the Show column for the counter you want to hide. Select the check box to add the counter to the display again.

You can also remove the counter from the display permanently by clicking the Remove button for the specific counter.

Creating a custom report from collected data

The Custom Reports pane allows you to generate three different kinds of reports based on the data that has been collected through the filters in the Consolidated Logs pane and the Performance Monitor pane. You can create the following types of custom report:

• Top Sites report, which lists the most accessed sites in the farm for the time period for which data has been collected.

• Slow requests report, which lists the requests with the longest response times.

• Failed requests report, which lists all requests that were not responded to by the farm.

You can filter data for your report by using the Custom Reports filter. The filter parameters are persistent between uses of the Custom Reports Filter dialog box within an SPDiag project, so when you close a project in SPDiag and then reopen it, the filter parameters you had previously defined will appear when you open the filter again.

Filter data for a custom report

1. In the Trends view, in the Custom Reports pane, click the Filter Reports button.

2. In the Custom Reports Filter dialog box, select the type of report you want to create from the Report Type menu.

3. If you want to create the report using all available data, click Ok to view the report and close the Custom Reports Filter dialog box, or click Apply to view the report without closing the Custom Reports Filter dialog box.

4. You can exclude records from the report by setting filter parameters. In the Hide entries based on this filter section, use the And/Or field to specify whether a filter statement is in addition to or in exclusion of the prior filter statements, the Field name field to select the record you want to add to the filter, the Operator field to set statement operators (such as greater than, equals or contains), and use the Value field to specify the value to use in the filter statement.

   Tip

   To remove a parameter from the filter, right-click anywhere in the row and click Delete.

   It is important to remember that, like the Merged Logs filter, the Custom Reports filter is exclusionary, and any records you select in a filter will be excluded from the report.

   Note

   You can use filters to explicitly select data by using negative operators. For example, if you have selected the Host field in the Field Name column, you can collect only records specific to the host SERVER1 by setting the following filter:

   • Field name: Host

   • Operator: Not Equal

   • Value: SERVER1

   The use of the Not Equal operator will exclude from the report all records in which the host name does not equal the specified value. You can add other OR statements with negative operators to the filter to explicitly view other records.

5. Click Ok to create the report and close the Custom Reports Filter dialog box. You can also click Apply if you want to create the report and leave the Custom Reports Filter dialog box open.

You can now view the report in the Custom Report pane.

Collecting performance counter data using Live Capture

In addition to collecting data from logs and performance counter files, SPDiag can also capture on-demand performance counter information by using Live Capture. The SPDiag Live Capture feature allows you to create a data collector set on the target server to record real-time performance counter data.

After the data collector set you created has been run on the target server, you can use the filter in the Performance Monitor pane to select and collect data from the data collector set. The Live Capture feature does not collect any data itself, but only allows you to create a collector set to generate a data file on the target server.

To collect data using Live Capture, follow the procedure below.

Create a data collector set using Live Capture

1. On the Data menu, click Live Capture.

2. In the Live Capture dialog box, in the Servers menu, select the farm server from which you want to capture data.

3. Once you have selected a server, a list of available performance counters appears in the field below. Select a counter that you want to capture, and click the Add button.

   Note that some counters have multiple instances, which will appear in the Instances of selected object field when the counter is selected. In this case, select the desired instance from that list and click the Add button. Repeat this process for each instance you want to add.

4. After you have added all the performance counter instances you want to capture, in the Capture Options section, set the Sample Interval to the number of seconds between samples. The default is five seconds.

   Warning

   If you select a smaller sample interval, a larger volume of data will be collected for the time frame you specified, and will take longer to collect.

5. In the Capture counters between section, use the top row of date and time fields to set the start time and date for the Live Capture, and the bottom row of date and time fields for the end time and date. These settings represent the local dates and times on the server.

6. If you want to overwrite a Live Capture that might be currently running on the target server, you can leave the Replace a currently running Live Capture on that server with this new definition check box selected.

7. Click Create to create and run the data collector set on the target server.

8. Click Close to close the Live Capture dialog box.

The data collector set you created will generate a .blg file that contains the output of the performance counters you specified. This data will then be available for collection through the Performance Monitor filter.

You can also manually start, stop and delete the last collector set you created on a farm server. This is useful if, for example, you want to stop and restart a running collector set. Note, however, that you cannot start an expired collector set. If you want to run a collector set with the same counters as an expired set, you must create a new collector set, or simply leave the Live Capture dialog box open if you want to consecutively run several instances of the same collector set.

Start, stop, or delete a collector set

1. On the Data menu, click Live Capture.

2. In the Collector Sets section at the bottom of the Live Capture dialog box, select the target server from the Servers menu.

3. To start the last collector set you created, click Start.

   To stop the last collector set you created, click Stop.

   To delete the last collector set you created, click Delete, then click Yes in the dialog box that appears.

4. Click Close to close the Live Capture dialog box.

For more information about collector sets, see "Scenario 3: Create a Data Collector Set from Performance Monitor" in the TechNet article Performance and Reliability Monitoring Step-by-Step Guide for Windows Server 2008 (https://go.microsoft.com/fwlink/?LinkId=141339).

Exporting data

You can export data from SPDiag in the form of reports. The available reports are Snapshot, currently displayed graphs, currently displayed custom report, and currently displayed events and logs.

When you export a snapshot, you will be asked if you want to update the snapshot before exporting. If the snapshot data has not been recently updated, you can click Yes to automatically update the data before exporting.

Export a report

1. On the Data menu, click Export.

2. In the Export Reports dialog box, select the check boxes for the reports you want to export.

3. In the Location field, either click Select folder to browse, or type the path to the appropriate export folder.

4. Click Export to export the files.

If you are delivering the exported data to a third party, you can compress the file to reduce its size using a commercially available data compression application. Microsoft Customer Support can use these files to help you to identify issues with your SharePoint farm.

The following table displays the files that SPDiag exports.