Message Analyzer Tutorial
Updated: May 20, 2016
This section begins with some background concepts about Microsoft Message Analyzer and then goes into several mini-tutorials or Getting Started Primers that will help you get started with using this unique tool. Links are provided throughout so that you can navigate to more information about the described features as needed.
Go To Procedures
To go directly to procedures that demonstrate how to use Message Analyzer, see the following topics:
Procedures: Quick Start
Procedures: Using the Network Tracing Features
Procedures: Using the Data Retrieval Features
Procedures: Using the Data Viewing Features
Procedures: Using the Data Filtering Features
Procedures: Using the Asset Management Features
Procedures: Using the Chart Viewer Layout Configuration Features
The overarching and new approach that Message Analyzer uses when capturing traffic is to limit network noise and to expose at top-level both the issues that occur at lower levels and hidden information that is critical to quick analysis. Message Analyzer does this by enabling you to remove lower-level details so you can focus on higher layer data of interest, by bubbling up message summaries and diagnostics to top-level transactions and operations (request/response pairs), and by locating message fragment reassemblies within the origins tree (stack messages) rather than in a dispersed chronological display. In this manner, the important information that you need to see for any particular message is readily exposed at top-level in the Analysis Grid viewer, which is the main analysis surface that Message Analyzer provides. Another significant feature that enables you to focus on messages of interest is Viewpoints, which display data from the perspective of a chosen protocol, module, or layer with no messages above it. For example, you could select a TCP Viewpoint and drive all TCP messages to top-level in the Analysis Grid to facilitate better analysis of TCP messages. This is in contrast to Message Analyzer's predecessor Network Monitor, which shows only flat or static message packets in original capture order and does not hide any noise, reassemble fragments, or simulate protocol behavior to allow for interpreting states and maintaining a protocol model, such as Message Analyzer does. Moreover, Message Analyzer formalizes its parser definitions to enable more artifacts to be derived from them, such as test cases and documentation.
You will learn more about these features in the next few sections that provide an overview of acquiring data through a Message Analyzer session and using various tools to focus data capture and analysis on specific types of data. After these sections, you can review the Getting Started Primers.
Message Analyzer enables you to capture, display, and analyze protocol messaging traffic, and to trace and assess system events, Windows component events, and device messages. It also provides the capability to retrieve, aggregate, and analyze data from one or more saved traces, which includes support for the .etl, .cap, .pcap, .pcapng, .tsv/.csv, .evtx, and .log input file formats, in addition to Message Analyzer native files in the .matp or .matu format, as described in Locating Supported Input Data File Types. If you work with text based .log files, Message Analyzer enables you to retrieve data from various common text .log file types with the use of built-in text log parsers that are described in Parsing Text Log Files. Also note that if you have a custom text .log file, an extensibility feature of the Microsoft Protocol Engineering Framework (PEF) enables Message Analyzer to retrieve its data with the use of a custom configuration file. However, you will need to create this file in order to fully parse your text log, as described in Parsing Text Log Files. Message Analyzer also enables you to extend the functionality of the graphic Charts feature so that you can create a custom data viewer to your own specifications, as described in Extending Message Analyzer Data Viewing Capabilities.
Message Analyzer makes use of two different types of sessions to acquire input data, as described in Starting a Message Analyzer Session. These consist of a Live Trace Session and a Data Retrieval Session, which provide data from the live capture of network traffic, events, system messages, and device messages; and saved traces, logs, and text logs, respectively. In a Live Trace Session, PEF provider-drivers and/or other system ETW Providers listen for and capture protocol messages and events at various stack layers or from other components. The messages and events are passed to the PEF Runtime where they are decoded by Open Protocol Notation (OPN) parsers and then temporarily saved in a Message Store. To access and display these messages, Message Analyzer consumes the PEF Runtime data, as described in the PEF Architecture Tutorial. Messages are displayed by default in the Analysis Grid viewer, where you can begin your data analysis process; however, other data viewers and various Tool Windows are also available to streamline message analysis.
Live Trace Session
In a Live Trace Session, you have the option to capture data from the local computer and/or multiple remote computers in concurrent subsessions that return all data to the common initiating live session that you configure with a chosen data viewer. Moreover, the local computer is the default host on which a Live Trace Session captures data; however, if you specify valid connection/authentication credentials for other remote computers, you can capture data simultaneously on those computers as well. Message Analyzer also provides you with the flexibility to run multiple concurrent Live Trace Sessions, optionally with each having different message provider and filtering configurations, to target different computers. You can do this by simply adding one or more Live Trace data sources in the New Session dialog, specifying the hosts from which to capture the data, and selecting or creating Session Filters, as described in Configuring Session Scenarios with Selected Data Sources.
Quick Tracing — to get started very quickly with a Live Trace Session, you can make use of Start Page features that enable you to start a new Local trace session at Link Layer or begin the configuration phase for a new session—with a single click—as described in Quick Session Startup.
To learn more about configuring a Live Trace Session, see Capturing Message Data.
Data Retrieval Session
In a Data Retrieval Session, Message Analyzer enables you to retrieve and aggregate saved message collections from multiple sources, including traces and logs, in any combination. This means you can mix and merge data from any of these sources and display it in the Analysis Grid or other selected data viewer. If you know that certain events of interest have occurred at a particular time in a collection of data sources, you can configure a Time Filter to view data in a window of time that you specify to eliminate extraneous data and improve performance. You can also set Time Shifts to accommodate for different time zones or skewed machine times across different data sources. You might also select or configure a Session Filter that enables you to return specific data that is based on the filtering criteria that you specify, while at the same time further improving performance.
To learn more about configuring a Data Retrieval Session, see Retrieving Message Data.
Although Message Analyzer enables you to capture messages from many system components, the PEF providers used by Message Analyzer enable you to capture data at several different layers, which provide unique inspection points into the protocol stack. For example, by specifying any Trace Scenario that uses the Microsoft-PEF-WFP-MessageProvider, you can focus on capturing messages above the IP/Network Layer by filtering out lower-level messages through the Windows Filtering Platform (WFP), upon which the Microsoft-PEF-WFP-MessageProvider is based. Message Analyzer also enables you to temporarily set a predefined Viewpoint that filters, reorganizes, and redisplays the data from the perspective of a selected protocol or module type, such as HTTP, TCP, SMB, or ETW, so that you can focus on specific message traffic that is defined by the Viewpoint, while removing all messages above the Viewpoint level to create a focused set of messages.
You can also select a predefined Parsing Level that controls the stack level to which Message Analyzer parses, while passing certain messages in these scenarios that are useful to your data analysis perspective, as described in Setting the Parsing Level. In addition, you can make use of Aliases, as described in Using and Managing Message Analyzer Aliases, to configure user-friendly names for cryptic field values; and you can take advantage of the Unions feature, described in Configuring and Managing Message Analyzer Unions, to correlate differently named fields that are of the same type in different data sources. You can even capture and analyze loopback traffic for local application communications that use the IPv4 or IPv6 loopback addresses, in the Loopback and Unencrypted IPSEC and Local Loopback Network Trace Scenarios, as described in Built-In Trace Scenarios.
You also have the option to select specific data that you want to isolate for focused analysis by making use of any of the following:
Fast Filter — a provider/driver-level filter that is very fast and efficient, as described in PEF-NDIS Fast Filters.
Keyword Filter — returns only the events from an ETW Provider that are defined by one or more designated event Keywords, as described in System ETW Provider Configuration Settings.
Session Filter — creates a focused set of trace results that is determined by filtering criteria, as described in Working with Session Filters.
Quick Filter — creates a window of time in which to view data, as described in Applying a Time Filter to Session Results.
View Filter — when applied to a set of trace results, passes only the message data that meets the filtering criteria that you specify, as described in Applying and Managing Filters. Enables you to create a focused set of results during an Analysis Session.
Furthermore, Message Analyzer enables you to decrypt data that is encrypted with the Transport Layer Security (TLS) and Secure Sockets Layer (SSL) protocols, for example Remote Desktop Protocol (RDP) and HTTPS messages. The Decryption feature also provides a Decryption Tool Window that presents summary and statistical data for the decryption session to facilitate analysis, as described in Using the Decryption Feature for TLS, SSL, and RDP.
These capabilities solve many inherent capture, data display, and analysis problems, such as the visibility of encrypted data, assessment of loopback traffic that is enabled by the Local Loopback Network scenario, and seeing traffic from the Viewpoint of a protocol. The underlying technologies that support Message Analyzer also machine-validate message structure and values, behavior, and architecture based on protocol specifications; and if errors occur, they are surfaced very quickly to top-level as diagnosis messages. To this end, Message Analyzer also provides a Diagnostics Tool Window that summarizes all the diagnostic messages in a trace, which interactively drives selection of corresponding messages in the Analysis Grid viewer.
Message Analyzer is also an effective tool for testing and verifying protocol implementations. See the Open Specifications documentation library for more information about protocol technical specifications.
The sections that follow provide brief conceptual tutorials that serve as getting started primers for Message Analyzer functionality. These tutorials correspond to the major tasks that you perform from the Message Analyzer user interface, where you can:
When capturing data live, Message Analyzer makes use of various message providers that focus on different layers or types of data. These providers are included in every Message Analyzer installation and consist of common Microsoft-PEF providers, the Microsoft-Windows-NDIS-PacketCapture provider, and various ETW Providers that exist on the Windows system by default. The providers are briefly described here.
Common Microsoft PEF Message Provider-Drivers — all PEF drivers are instrumented with Event Tracing for Windows (ETW) provider technology, which enables them to take advantage of the ETW event tracing, buffering, logging, and event delivery infrastructure. In addition to numerous system ETW providers and other message capture components, all Message Analyzer installations contain the following PEF provider-drivers, the configurations for which are accessible after you select a Trace Scenario from the Select a trace scenario drop-down list on the Live Trace tab of the New Session dialog for a Live Trace Session.
Some of the message providers described in this section may be different than what you find on your computer, because of an operating system version dependency. For example, on computers running the Windows 7, Windows 8, or Windows Server 2012 operating system, the Microsoft-Windows-NDIS-PacketCapture provider does not exist for the Local Network Interfaces Trace Scenario. Instead, the Microsoft-PEF-NDIS-PacketCapture provider in the Local Network Interfaces scenario is included for that purpose on those computers. On computers running the Windows 8.1, Windows Server 2012 R2, and Windows 10 operating systems, the Microsoft-Windows-NDIS-PacketCapture provider is installed as part of the operating system.
Microsoft-PEF-NDIS-PacketCapture provider — an ETW-instrumented, Network Data Interface Specification (NDIS) light weight filter (LWF) driver that captures Ethernet frames at the Link Layer and delivers them to Message Analyzer through the ETW infrastructure. Also includes the capability to configure Fast Filters that operate efficiently at the driver-level to isolate specific message types, thereby passing less data and reducing system loads and resource consumption.
To learn more about the Microsoft-PEF-NDIS-PacketCapture provider, see Microsoft-PEF-NDIS-PacketCapture Provider.
Microsoft-PEF-WFP-MessageProvider — an ETW-instrumented filter driver that is based on the Windows Filtering Platform (WFP). It captures message traffic above the IP/Network Layer and delivers that traffic to Message Analyzer through the ETW infrastructure. This provider also enables you to configure Fast Filters to isolate specific messages of interest and improve trace performance. This provider is now enabled for remote capabilities when capturing data on remote Windows 10 computers only. In addition, you can set the Select Discarded Packet Events option when configuring this provider to log discarded packets.
To learn more about the PEF-WFP provider, see Microsoft-PEF-WFP-MessageProvider.
Microsoft-PEF-WebProxy — an ETW-instrumented provider that uses the Fiddler API and acts as an HTTP proxy to intercept and capture all HTTP traffic to and from a client web browser in unencrypted format. Also provides the capability to configure driver-level Hostname and Port filters to isolate specific messages and improve performance.
To learn more about the PEF-WebProxy provider, see Microsoft-PEF-WebProxy Provider.
Microsoft-Windows-NDIS-PacketCapture provider — an ETW-instrumented provider that has remote capabilities along with special NDIS stack and Hyper-V-Switch extension layer filtering, adapter configurations, packet traversal path directivity, and other filters and specifiers that you can configure.
The Microsoft-Windows-NDIS-PacketCapture provider with remote capabilities is used on the Windows 8.1, Windows Server 2012 R2, and Windows 10 operating system only, as described in Built-In Trace Scenarios.
To learn more about capturing messages from one or more remote hosts and configuring the Microsoft-Windows-NDIS-PacketCapture provider, see Capturing Data Remotely.
System ETW providers — write events for various components on your system that have been instrumented as ETW event providers. This includes ETW providers that define their events with the use of the following:
You can specify the message providers that you want to use to capture data from the network or other components by configuring a Live Trace Session, as shown in the figure that follows.
Figure 2: Message Analyzer Live Trace Session configuration
Predefined provider configurations are contained in the built-in Trace Scenarios that you can select from the Select a trace scenario drop-down list on the Live Trace tab of the New Session dialog. These Trace Scenarios are templates that contain predefined message provider configurations that are tailored for capturing data from various components and at different stack layers. Optionally, you can enhance the scope of data retrieval by adding other system ETW providers to the scenario. Also, if you have created and saved any custom Trace Scenarios in the Message Analyzer Trace Scenario asset collection Library by using the Save Trace Scenario feature, these are also available for selection from the Select a trace scenario drop-down list. You can also modify the capture configuration of PEF and other ETW Providers from the Live Trace tab of a New Session to isolate specific message traffic and realize performance enhancements.
For example, by clicking the Configure link for a selected message provider in the ETW Providers list on the Live Trace tab, you can display a configuration dialog and specify Fast Filters that work very efficiently at the kernel level. These low-level filters enable you to quickly retrieve specific messages that meet the filtering criteria that you specify, which reduces the scope of the data to be returned by the trace. In turn, this accelerates data capture and minimizes the parsing time. You also have the option to select or create a Session Filter for a Live Trace Session (or a Data Retrieval Session) to reduce the scope and count of messages that you retrieve, and as a result realize performance improvements. The difference between a Fast Filter and a Session Filter is that Fast Filters work at the provider/driver level and are therefore not subject to the Runtime parsing process, which makes them faster, whereas Session Filters are subject to parsing, which makes them a little slower.
Other ETW Provider settings that you can configure for a Live Trace Session consist of the following:
System Network adapter filters and logically ANDed Fast Filter group settings — the configuration is accessible from the Provider tab of the Advanced Settings – Microsoft-PEF-NDIS-PacketCapture dialog for Local Network Interfaces Trace Scenarios, as described in Using the Advanced Settings - Microsoft-PEF-NDIS-PacketCapture Dialog.
Advanced filters — includes settings for NDIS stack filters; extension layer filters for Hyper-V-Switches that service virtual machines (VMs); and Direction (packet traversal), EtherType, IP Protocol Number, MAC Address, and IP Address filter settings — the configuration is accessible from the Provider tab of the Advanced Settings – Microsoft-Windows-NDIS-PacketCapture dialog for the Local Network Interfaces Trace Scenario, as described in Using the Advanced Settings - Microsoft-Windows-NDIS-PacketCapture Dialog.
WFP Layer Set and Fast Filter settings — the configuration is accessible from the Provider tab of the Advanced Settings - Microsoft-Pef-WFP-Message Provider dialog, as described in Filtering Live Trace Session Data.
Hostname and Port Filter settings — the configuration is accessible from the Provider tab of the Advanced Settings – Microsoft-Pef-WebProxy dialog for the Pre-Encryption for HTTPS Trace Scenario, as described in Filtering Live Trace Session Data.
Keyword event and error Level filters — the configuration is accessible from the ETW Core tab in the Advanced Settings of all provider configuration dialogs; however, not all ETW Providers make Keyword and Level filter settings available, as some providers are not instrumented with them. See System ETW Provider Configuration Settings for additional details.
You can very quickly start capturing data with Message Analyzer by clicking either of the following on the Message Analyzer Start Page:
Start Local Trace button — starts a local trace with the NDIS provider.
Favorite Scenarios list item — starts a local trace with the Local Network Interfaces, Loopback and Unencrypted IPSEC, or Pre-Encryption for HTTPS Trace Scenarios.
To learn more about configuring a Live Trace Session, see Capturing Message Data.
To learn more about usage configurations for PEF-based providers and other message providers, see the Built-In Trace Scenarios topic.
Message Analyzer provides several global options that enable you to specify certain default values or make default selections that can affect Message Analyzer performance, display configurations, or feature activation. For example, you can specify a default Session Viewer, the default configuration for Text Log Files, Time Display format, Decryption certificate data, Parsing options, preview Features, and so on. You can set these options at any time; however, you would typically do so prior to starting a Live Trace Session or a Data Retrieval Session.
To learn more about the global Message Analyzer options that you can set, see Setting Message Analyzer Global Options.
Message Analyzer can display message traffic that is captured from specific protocol modules only if the protocol object model (POM) repository within the PEF architecture contains compiled OPN descriptions representing the architecture, behavior, and data for those protocols. Message Analyzer ships with OPN descriptions for a large number of protocols, such as Microsoft Windows and other common public protocols, in addition to Office, Exchange, SharePoint, and SQL protocols. This enables you to capture a wide array of network protocol and application messages. In addition, to support your data analysis process, Microsoft makes Protocol Technical Specifications available on the Microsoft Developer Network (MSDN) web site, while you can find other standard RFC protocol specifications on the Internet.
You can use the technical documents (TDs) provided by Microsoft as references that depict protocol architecture, behavior, and data, as it was designed, to facilitate analysis of the messages you capture with Message Analyzer. For example, you could verify the value of a particular field or confirm the presence of required parameters for a particular method of a specific protocol that is failing to perform properly, although Message Analyzer has a built-in message validation feature that does this automatically.
Event tracing functionality is integrated with all message providers that are used by Message Analyzer. Moreover, all Message Analyzer providers are instrumented with ETW technology so that events can be returned in a trace along with network traffic. The Message Analyzer trace model uses ETW to enable integrated capture and display of messages and events from a large number of system components. Whenever you start a Live Trace Session, the underlying message provider/s in the Trace Scenario that you select are enabled to an ETW Session Controller, which determines if there are any specific Keyword event or error Level settings that modify which events are to be returned to the ETW Consumer, which in this case is Message Analyzer. If there are no such settings, then the ETW Session Controller returns all events generated by the component that is instrumented for ETW. Message Analyzer then displays detailed, human-readable information for events at an event layer that is below the networking stack in all traces.
For this to be possible, OPN must be generated for any manifest-based system ETW Provider that you employ in a Live Trace Session so that ETW events can be properly parsed by the PEF Runtime. To generate the OPN, manifests for system ETW Providers in use are retrieved so that OPN descriptions can be inferred from them to provide the basis for Message Analyzer to successfully parse event structures. To facilitate this process, the PEF architecture contains an ETW Manifest Import Adapter. This is a protocol object model (POM) adapter that converts an ETW manifest for a given ETW Provider into a POM model, and then publishes it to the PEF Runtime so it can parse and dispatch ETW messages generated by that provider. The OPN actors and endpoints that enable parsing and dispatching messages for an ETW Provider that you specify in a Live Trace Session are dynamically generated at runtime by the ETW Manifest Import Adapter.
In a future Message Analyzer release, it may be possible to extend your system with additional system ETW Providers from which Message Analyzer can receive events. To do this, you would need to create an OPN configuration file for each new ETW Provider so that the associated provider manifests imported into the system can correctly infer an OPN description that provides proper parsing of ETW events.
An ETW Provider manifest defines the event descriptions and format in which events are written by the provider. To create an OPN configuration file for a new system ETW Provider, you will need an OPN Configuration Guide that may be developed for a future Message Analyzer release. Until this occurs, you might consider one of the many system ETW Providers that are accessible from the Add Provider drop-down list on the Live Trace tab of the New Session dialog.
Message Analyzer also enables you to modify certain aspects of ETW Sessions to focus on capture of specific events and/or to improve performance as follows:
ETW Provider — you can specify the events that you want to receive from a system ETW Provider by configuring Keyword and/or Level filtering. You can configure Keyword and Level filters from the ETW Core tab in the Advanced Settings dialog for the particular message provider that underlies the Trace Scenario that you selected, as described in Configuring a Live Trace Session, that is, for system ETW Providers that permit Keyword and Level filter configuration. Configuring system ETW Provider filtering for event tracing enables you to decrease the event volume and capture time by isolating specific types of events to retrieve in the trace, rather than all events, and enables you to focus your analysis on specific events that you choose.
ETW Session Configuration — you can configure certain aspects of the underlying ETW Session in which an ETW Provider participates to enhance session performance. This mainly involves adjusting settings for the ETW buffer configuration of the ETW Session that is managed by an ETW Session Controller. These adjustments are available from the Message Analyzer ETW Session - Advanced Configuration dialog that is accessible by clicking the ETW Session Configuration button on the Live Trace tab of the New Session dialog, as shown in the previous figure.
To learn more about optimizing an ETW Session, see Specifying Advanced Session Configuration Settings.
To learn more about how system ETW Providers function in the ETW framework, see the ETW Framework Tutorial.
To learn more about configuring system ETW Providers, see Adding a System ETW Provider and System ETW Provider Configuration Settings.
Message Analyzer also supports registered event providers on your system that use the managed object format (MOF) schema as the basis of generating their events. These event providers typically appear in the Add Provider drop-down list on the Live Trace tab of the New Session dialog for a Live Trace Session. With MOF support, events that are captured by Message Analyzer from MOF-instrumented providers can be fully parsed. Without MOF support, messages that are captured from MOF-based providers are displayed as simple ETW messages with no additional parsing.
To provide support for MOF-instrumented providers, including fully parsing events from such providers, Message Analyzer uses an extension to the existing ETW adapter. This adapter normally handles ETW providers that have a manifest that is created at the time the provider is instrumented for ETW. When an ETW event arrives, Message Analyzer checks to see whether an OPN description exists that can parse the event. If an OPN description cannot be found, then Message Analyzer attempts to retrieve the manifest-based event schema, from which it can generate OPN. In a similar manner, Message Analyzer does the following to support MOF when events arrive:
Verifies whether events are generated by an MOF-based provider.
Checks the local system for an existing OPN description that can parse the events.
Uses the extended version of the ETW adapter to generate an OPN description based on the MOF schema of the provider, if an existing OPN description was not found.
Detecting MOF Schema
In Message Analyzer, there are typically three sources from which MOF events can derive, including live traces, saved trace files such as the native Message Analyzer parsed format (.matp), and saved trace files in other supported formats such as .matu, .etl, and .cap. As previously indicated, if there is an existing OPN module (see Protocol Modules) that can consume the events, then the events are parsed according to the OPN description and background generation of OPN is not required. However, if there is no existing OPN module to parse the events, Message Analyzer then attempts to locate the MOF schema as follows:
Live trace — when you run a Live Trace Session that utilizes MOF-based event providers, the locally installed MOF schemas are retrieved from the appropriate event provider/s that are installed on the local machine, and OPN descriptions for the provider events are automatically generated for parsing the event fields.
Saved .matp files — if one or more MOF schemas were used to parse messages from an MOF provider when a trace is taken with Message Analyzer, the schemas become part of the .matp trace file when it is saved in the same format. The schema is thereafter provided to Message Analyzer at the time the .matp trace file is loaded, making it independently available to facilitate event parsing whether or not MOF schemas exist on the local system or were deployed during Message Analyzer installation.
Saved non-.matp files — these files will not contain the embedded schema information, therefore Message Analyzer looks up local files deployed during installation. If a local .mof file is discovered, it is used as the MOF schema from which an OPN description is generated for parsing events. Otherwise, the system MOF schema is retrieved and used in a similar manner.
If Message Analyzer requires a MOF schema for a provider that is installed on the local system and cannot find one, then Message Analyzer will display simple ETW messages only, with minimal parsing for that provider’s messages.
Deploying a Custom MOF Provider
If you have a custom MOF-based provider that you want to deploy on your local system, you can use the WMI compiler tool mofcomp.exe to register your provider and its MOF schema. Thereafter, Message Analyzer will be able to locate the MOF schema, should an OPN description need to be created to parse the MOF-based events of the provider. You will find the mofcomp.exe tool in the following directory on your computer:
To learn more using the mofcomp.exe tool, see mofcomp in the WMI Command Line Tools topic on MSDN.
When you start a Data Retrieval Session, the configuration of which is shown in the figure that follows, you can load data from saved trace files and logs into Message Analyzer, which includes .matu, .matp, .etl, .cap, .pcap, .log files, and others, as described by the table in Locating Supported Input Data File Types. After clicking the Add Files button on the Files tab in the New Session dialog for a Data Retrieval Session, you can navigate to target files that contain the data you want to load into Message Analyzer. After the files containing the target data display on the Files tab, you can also specify subsets of those files in your Files list to create message collections that target specific data to be loaded into Message Analyzer and parsed. To create a subset, you simply select the check box to the left of the file that contains the data you want to load. Note that a Data Retrieval Session enables you to aggregate and merge message data from multiple data sources that include various types of log files and traces.
Figure 3: Message Analyzer Data Retrieval Session configuration
You can also select specific data to retrieve from a target message collection while blocking all other messages that do not meet the filtering criteria that you define with a Session Filter or Time Filter. A Session Filter narrows the scope of data retrieval to only the message types that meet the criteria of a filter that you manually define, or one that you select from the centralized filter Library. A Time Filter enables you to specify a window of time in which to view data in a correlated target message collection that can consist of one or more sources from which you load data into Message Analyzer.
To learn more about configuring a Data Retrieval Session, see Retrieving Message Data.
If you have a text-based log file that contains log entries in a unique format, PEF architecture enables you to load and view the data from the log file, but you might need to create an OPN configuration file to drive the process. However, note that the configuration file type that you need might already exist in the Text Log Configuration column drop-down list that appears below the toolbar on the Files tab of the New Session dialog for a Data Retrieval Session. This drop-down list is populated with common built-in configuration files that you can select only after you click the Add Files button and retrieve a *.log file that contains the data you want to load into Message Analyzer.
Built-In OPN Configuration Files
The built-in configuration file types that are currently available for selection are described in the list that follows. A short description of the purpose of each configuration file type is included:
AzureStorageClientDotNetV4 — provides the OPN configuration that parses Azure .net client storage logs.
AzureStorageLog — provides the OPN configuration that parses Azure .log files that are saved in BLOB containers.
CheckSymCSV — provides the OPN configuration that parses the CSV output of the Exchange CHECKSYM utility, which is commonly used to perform file version and checksum comparisons of binaries and configuration files.
Cluster — provides the OPN configuration that parses event cluster text logs.
Dcdiag — provides the OPN configuration that parses the output of the Domain Controller Diagnostics Tool (Dcdiag).
DPMRegistry — provides the OPN configuration that parses special registry output text logs for the Data Protection Manager (DPM) component.
EventLogCSV — provides the OPN configuration that parses traces that are exported as a CSV file, but with more value than a regular CSV file.
IIS — provides the OPN configuration that parses text logs generated by web servers.
LSP — provides the OPN configuration that parses text logs generated by the Local Security Authority (LSA) component, which applications can use to authenticate and log users on to the local system. The log files provide access to some data in clear text that is otherwise encrypted by messages on the wire. Administrative privileges are required to view these logs.
LYNC — provides the OPN configuration that parses UCCAPI logs from the Lync client application.
Netlogon — provides the OPN configuration that parses logs for diagnosing logon issues on domain controllers.
SambaSysLog — provides the OPN configuration that parses text logs generated on Samba Linux machines.
SCCM — provides the OPN configuration that parses System Center logs.
SQLServerError — provides the OPN configuration that parses SQL Server error logs.
SQLServerSetup — provides the OPN configuration that parses SQL Server setup logs
ULS — provides the OPN configuration that parses SharePoint logs.
VMM — provides the OPN configuration that parses System Center Virtual Machine Manager logs.
DefaultSimpleLogFileReaderConfig — a generic configuration file that can parse most text logs, for example those from a domain controller.
The listed text log configuration files are contained in a Message Analyzer Device and Log File Version 1.3 asset collection that you can configure for automatic downloads and updates from a Microsoft web service through the Sharing Infrastructure. The management features for this and all Message Analyzer asset collections are available from the Asset Manager dialog, which is accessible from the global Message Analyzer Tools menu.
To learn more about managing Message Analyzer asset collections, including downloading and auto-syncing any collection for automatic updates, see Managing Message Analyzer Assets.
If none of the built-in text log configuration files meet your requirements, then you can create a new one that is specifically designed to parse the data in your text log, as described in Opening Textual Log Files. Whenever you create a new configuration file for a text log, it is added as an item to the Text Log Configuration drop-down list below the toolbar on the Files tab of the New Session dialog. It is also added to the Default text log configuration drop-down list in the Text Log Files pane on the General tab of the Options dialog, which is accessible from the global Message Analyzer Tools menu. From the latter drop-down list, you have the option to set a specific configuration file as the global default for all text log files from which you will load data into Message Analyzer. This makes it convenient if you work with a particular type of .log file consistently.
Configuration File Contents
A configuration file contains a description of the log's messages in OPN and RegEx notation, which ensures that text log data that is loaded into the system can be properly parsed and then displayed in Message Analyzer. The text-based log data is loaded into the Runtime through a Log File Adapter and the OPN configuration file drives the process. The message definitions contained in the OPN configuration file are compiled by the OPN Compiler to confirm the validity of the configuration file and the integrity of the OPN description that will reside in the POM.
When you create an OPN configuration file, you need to identify each unique log entry and map it to a message structure. You can do this with RegEx notation, which is designed for matching strings of text. RegEx provides the functionality you will need to match data through the mechanism of capture variables, which you can use to map extracted log file data to message fields.
Message Analyzer supports loading regular comma-separated-value (CSV) and tab-separated-value (TSV) data file formats directly, without the need for an OPN configuration file.
To learn more about how to create an OPN configuration file, download the OPN Configuration Guide for Text Log Adapter document.
To learn more about other OPN configuration file requirements, see the Addendum 1: Configuration Requirements for Parsing CustomText Logs topic.
In Message Analyzer, messages are reassembled as part of the PEF Runtime parsing process. The message data is then displayed in a Message Analyzer data viewer that you select, for example, the Analysis Grid viewer, which is shown in Figure 5 of the Filter Message Data section and described in the Analysis Grid Viewer topic. The Analysis Grid viewer is the primary analysis surface in Message Analyzer. It displays message data as expandable top-level parent nodes that contain all the child node (message stack) messages and message fragments that were involved in a particular transaction or operation.
By default, the Analysis Grid viewer hides message fragments in the message stack (for example, TCP virtual segments), while presenting top-level messages such as request/response message pairs for a particular transaction in a single expandable top-level message node that is known as an operation. By organizing messages this way, you can easily determine such important values as the ResponseTime, which can tell you how long it is taking to receive the first server response to a request message, without having to search through hundreds of messages to find it. This is important to analysis because it can indicate how long a service is taking to respond, which can rule out network issues. Another important value is the ElapsedTime, which can tell you how long an operation is taking to complete, including all the associated message fragments. Also, by performing a Message Analyzer sort on ElapsedTime, you can determine which operations (with fragments) took the longest to complete. In addition, by sorting on ResponseTime, you can determine which request/response pairs had the longest response times.
Grouping Messages in the Analysis Grid Viewer
Another feature that is important to data analysis is the Group feature. By right-clicking selected Analysis Grid viewer columns in succession and selecting the context menu Group item for each one, you can create a data display of nested groups that provides a convenient way to organize and explore targeted trace data. As an additional example of grouping, you could create IPv4 Network and TCP Transport groups in the Analysis Grid viewer by executing the Add as Grouping command from the right-click context menu of the Field Chooser Tool Window to Group your data into these field-categories. This quickly organizes your data into groups of IP conversations that took place across a trace, with the TCP ports that supported those conversations nested within each IP group, resulting in a unique analysis perspective.
Grouping Messages in the Grouping Viewer
You can also make use of the Grouping viewer, which has a set of built-in View Layouts that render Analysis Grid message data into predefined nested Group configurations to create unique analysis contexts. You can also create and save your own Grouping View Layouts that you customize to your environment based on message fields that you select from the Field Chooser Tool Window. The Grouping viewer is accessible from the Common category of the New Viewer drop-down list that appears on the global Message Analyzer toolbar.
With the Grouping viewer, you can organize your traffic into summary hierarchies based on built-in or custom-designed Grouping View Layouts that are configured with message field groups in nested configurations. You can also manually adjust (pivot) your group layout by dragging and dropping Group labels to obtain different message correlation configurations that result in unique analysis contexts. Advantages of viewing data with the Grouping viewer include the following:
Organize data into unique hierarchies to expose targeted information that you can quickly extract from large data sets, which can otherwise be difficult to do.
Arrange nested Group configurations so you can isolate messages of interest at defined levels and drill down into the nested Groups to obtain a concise analytical focus.
Locate the Group(s) with the highest message volumes for performance assessment.
Note that every Message Analyzer installation provides a default Message Analyzer Grouping View Layouts asset collection that appears in the Asset Manager dialog, where you can manage the download and auto-sync status of the collection.
To simplify troubleshooting, Message Analyzer provides the Viewpoint Tool Window that enables you to examine network traffic from the perspective of a protocol. By setting a predefined Viewpoint, you can focus on specific messages at top-level in the Analysis Grid viewer with no layers above them. Moreover, because the Viewpoint temporarily removes all messages above the applied protocol Viewpoint, only the protocol messages associated with the applied Viewpoint appear at top-level in the Analysis Grid viewer. This feature is advantageous when you have higher-layer traffic that obscures the underlying messages that you want to troubleshoot. Note that every Message Analyzer installation provides a built-in Message Analyzer Viewpoints asset collection that appears in the Asset Manager dialog, where you can manage the download and auto-sync status of the collection.
Viewing Message Details
You can obtain a full visual representation of message details in the Analysis Grid viewer, including field names and values, by double-clicking any top-level parent message node or nested child message node. This information is presented inline on a Fields tab with other selectable data tabs that can include the Stack, Diagnosis, and Embedded tabs. You can also select any message and view its details data in a separate window that is called the Details Tool Window, which includes field, value, and payload data. Other Tool Windows are also available to enhance your data analysis perspective, for example, the Message Data, Field Data, Diagnostics, and Decryption windows. You can also view stack information in a separate window known as the Message Stack Tool Window, which provides an alternate view of the origins tree (message stack) below any top-level message that is normally hidden by collapsed message nodes in the Analysis Grid viewer. Note that many Message Analyzer Tool Windows are interactive, because they either drive or are driven by message or data selection in other windows. For instance, by selecting a field in the Details window, the Message Data window immediately snaps to the selection and highlights the corresponding hexadecimal value of the selected field.
Viewing Data from Multiple Sessions
Message Analyzer also provides session viewer functionality from the Session Explorer Tool Window, to enable you to easily explore the data in different types of session data viewers, which can include viewers with Charts that employ top-level summaries in graphic and tabular formats, a Pattern Match viewer, and viewers that have additional timeline-oriented statistics for protocol communications or other messages and events. The Session Explorer window is located in the right sector of the Message Analyzer user interface.
By right-clicking a session node in Session Explorer, you are presented with the New Viewer context menu item, which displays a drop-down list that enables you to select other data viewers that display data in separate session viewer tabs. Thereafter, any new data viewer that you specified is listed and uniquely identified in the Session Explorer window navigation area. If you select any Session Explorer node, Message Analyzer responds by immediately displaying the data on the session viewer tab that corresponds with the selected node.
The figure that follows shows an example of a Chart viewer known as the IP/Ethernet Conversations by Message Count that you can select from the New Viewer drop-down list, which is accessible from the Session Explorer context menu. This Chart-style viewer contains a summary of the top eight IP/Ethernet conversations (IP or Ethernet addresses) along with various tabular statistics such as message Count in each conversation, total payload volume in Bytes of messages in each conversation, Duration of each conversation, BPS (bits-per-second) data transmission rate, and so on. For further details about this viewer, see the IP/Ethernet Conversations by Message Count topic.
Figure 4: Message Analyzer IP/Ethernet Conversations by Message Count viewer Layout
Message Analyzer data viewers are not integrated, in that changes you make to data in one viewer, for example with filtering, are not reflected in other viewers that are displayed for the same session. In a future Message Analyzer release, filtering controls may be included with each type of data viewer to emphasize that the effects of filtering apply only to the data viewer where a Filter Expression is executed.
Also, some Message Analyzer data viewers are interactive, in that data selection in one viewer drives the display of data in another viewer (or Tool Window). For example, you can double-click a bar chart column or a timeline module node in the Protocol Dashboard viewer that represents the messages of a particular protocol that were captured in a trace, and display only those messages in a new Analysis Grid viewer tab for data assessment purposes. You might do this, for example, to isolate a group of messages that reflect a high message count for a particular protocol or module to expose any underlying issues.
To learn more about the Message Analyzer data viewer infrastructure, see Data Viewer Concepts.
To learn more about Message Analyzer data viewers that you can work with during data analysis, including the Chart viewers, see Data Viewers.
To learn more about Viewpoints and operations, see Applying and Managing Viewpoints.
To learn more about the Analysis Grid viewer Group feature, see Using the Analysis Grid Group Feature.
To learn more about the Grouping viewer, see the Grouping Viewer topic.
To learn more about Message Analyzer Tool Windows, including the Session Explorer window, see the Tool Windows topic.
To learn more about the Field Chooser, see Using the Field Chooser and the Field Chooser Tool Window topics.
Message Analyzer provides numerous filtering capabilities to enhance data retrieval, capture, and assessment processes. Filtering is critical for focusing on specific messages and enhancing performance. For example, if you were unable to filter message data in a Live Trace Session, you might need to examine potentially tens of thousands of messages to isolate a specific problem. What most Message Analyzer users need to observe is usually related to a specific protocol, error message, conversation, or process. By providing the ability to filter while retrieving, capturing, or viewing data, Message Analyzer provides a convenient way to reduce the scope of the data that you are working with and more effectively pinpoint your issues.
Using a Session Filter
When capturing data or loading data into Message Analyzer through a Live Trace Session or a Data Retrieval Session, as shown in Figures 2 and 3 of the sections Configuring a Live Trace Session and Retrieve Message Data, respectively, you can use the common Session Filter feature to isolate data that you want to work with from the live trace or saved message collection. However, you should carefully note that you can never recapture the data that you filter out with a Session Filter in a Live Trace Session, whereas with a Data Retrieval Session, you can always click the Edit Session button on the global Message Analyzer toolbar to return to session configuration, where you can remove or recast your filtering criteria and then reload the data from the originally specified saved files.
For instance, when configuring a Session Filter, you can specify a Filter Expression that isolates messages to a specific network address, port, or protocol, or that contains a particular field value or other text. Similarly, you can apply a View Filter or Color Rule to streamline the examination of data in the Message Analyzer default Analysis Grid viewer, which is shown in the figure that follows.
Figure 5: Message Analyzer Analysis Grid viewer
Using Special Filters for a Live Trace
You also have the option to use many other types of filters in a Live Trace Session, depending on the Trace Scenario and operating system you are running, as follows:
Fast Filters and WFP Layer Set filters — accessible from the Provider tab of the Advanced Settings – Microsoft-PEF-WFP-MessageProvider configuration dialog. You can display this dialog by clicking the Configure link to the right of the Microsoft-PEF-WFP-MessageProvider listing on the Live Trace tab of the New Session dialog after you select one of several Trace Scenarios that contain this provider from the Select a trace scenario drop-down on the Live Trace tab. For example, the Network Tunnel Traffic and Unencrypted IPSEC, Loopback and Unencrypted IPSEC, and Local Loopback Network Trace Scenarios all use the Microsoft-PEF-WFP-MessageProvider.
Fast Filter Groups and Adapter filters — accessible from the Provider tab of the Advanced Settings – Microsoft-PEF-NDIS-PacketCapture configuration dialog. You can display this dialog by clicking the Configure link to the right of the Microsoft-PEF-NDIS-PacketCapture listing on the Live Trace tab of the New Session dialog after you select the Local Network Interfaces Trace Scenario from the Select a trace scenario drop-down list on the Live Trace tab. The Microsoft-PEF-NDIS-PacketCapture provider is available on computers running the Windows 7, Windows 8, or Windows Server 2012 operating system only.
HostName and Port filters — accessible from the Provider tab of the Advanced Settings – Microsoft-Pef-WebProxy configuration dialog. You can display this dialog by clicking the Configure link to the right of the Microsoft-Pef-WebProxy provider listing on the Live Trace tab of the New Session dialog after you select the Pre-Encryption for HTTPS Trace Scenario from the Select a trace scenario drop-down on the Live Trace tab.
Event Keyword and error Level filters — accessible from the ETW Core tab of any Advanced Settings dialog for any Trace Scenario that you select from the Select a trace scenario drop-down list. You can display this dialog by clicking the Configure link to the right of any provider listing on the Live Trace tab of the New Session dialog. Note that not all ETW Providers contain an event Keyword configuration.
NDIS stack, Hyper-V-Switch extension layer, and Host adapter filters — accessible from the Provider tab of the Advanced Settings – Microsoft-Windows-NDIS-PacketCapture provider configuration dialog. You can display this dialog by clicking the Configure link to the right of the Microsoft-Windows-NDIS-PacketCapture provider listing on the Live Trace tab of the New Session dialog after you select the Local Network Interfaces, Remote Network Interfaces, or Remote Network Interfaces with Drop Information Trace Scenario from the Select a trace scenario drop-down list on the Live Trace tab. The Microsoft-Windows-NDIS-PacketCapture provider has remote capabilities and is available on computers that are running the Windows 8.1, Windows Server 2012 R2, or Windows 10 operating system only.
The filters that are available for the Microsoft-Windows-NDIS-PacketCapture provider in these scenarios consist of advanced driver-level filters that include the following:
Host adapter filters
If you want to isolate traffic to a particular virtual machine (VM) that is serviced by a Hyper-V-Switch, you should select the VM adapter in the Interface Selection section of the Advanced Settings - Microsoft-Windows-NDIS-PacketCapture dialog to select the adapter and then specify the MAC address of the VM adapter in the MAC Addresses box of the dialog, rather than simply selecting the adapter. Otherwise, you will return all switch traffic rather than the traffic of a selected VM, given that a Hyper-V-Switch driver cannot distinguish between VMs.
NDIS stack and Hyper-V-Switch extension layer filters
Packet traversal direction filters
IP protocol number filters
MAC address filtering
IP address filters
To learn more about these filtering capabilities, see Using the Advanced Settings - Microsoft-Windows-NDIS-PacketCapture Dialog.
Quick Filter — you can utilize a Quick Filter to configure a window of time in which to view the results of a Live Trace Session. This is particularly useful if you can approximate a time frame in which you suspect a particular issue occurred that you need to detect. The major advantage of using a Quick Filter is that you can remove it, modify the time window, reapply it, and repeat this process as many times as needed. You can access the Quick Filtering dialog by clicking the Edit command in the Quick Filter drop-down menu that appears in the global Message Analyzer Session menu.
Using View Filters to Manipulate a Set of Trace Results
After you capture or retrieve your message data in a Live Trace Session or Data Retrieval Session, respectively, you have a baseline set of trace results to work with. However, it is very likely that to analyze the data, you will want to manipulate it with various Message Analyzer tools to isolate specific messages of interest that can expose issues you are trying to detect. One of the most common ways to do this, is to use a View Filter to filter for data that is relevant to the problem you are trying to solve while filtering out data that isn't. This enables you to create a set of messages that is focused on the data you need to examine, without the encumbrance of scrutinizing potentially hundreds if not thousands of messages that are irrelevant to the issue at hand. When you apply a View Filter, the original data set is always preserved and re-displays after you remove any applied View Filter. Note that the effects of a View Filter apply to the in-focus data viewer only and do not impact other viewers, even in the same session.
The controls for locating, applying, and removing View Filters are located in the View Filter Tool Window, as described in Applying and Managing Filters. Note that you can select a built-in View Filter from the Message Analyzer Filters asset collection Library that exists in multiple locations, including on the toolbar of the View Filter Tool Window. You can also create your own custom View Filters that you can save to this Library for future use and sharing with others, but you will need to understand the Message Analyzer Filtering Language to create one (see link immediately below).
To learn more about the functionality of the built-in View Filters, see the Filtering Live Trace Session Results topic, which describes each filter in the centralized Filter Expression Library.
To learn more about the Filtering Language, see Writing Filter Expressions.
To learn more about auto-syncing, downloading, and managing the Message Analyzer Filters asset collection, see the Sharing Infrastructure and Downloading Assets and Auto-Syncing Updates topics.
Centralized Filter Expression Library
To apply a Session Filter, View Filter, Find Message filter, or Viewpoint filter, you will need to select a built-in Filter Expression from the centralized Library or you can manually create one. This Library contains the built-in Filter Expressions that are provided by the Message Analyzer Filters asset collection in every Message Analyzer installation, for which you can use the following for the indicated purpose:
Asset Manager dialog — to manage downloads and auto-sync updates for the Message Analyzer Filters asset collection.
Manage Filters dialog — to export and import asset collection items to and from others, respectively, for mutual sharing.
The Message Analyzer Filters asset collection Library is accessible from the following locations:
In the New Session dialog when configuring a Session Filter for a Live Trace Session or Data Retrieval Session.
On the toolbar of the View Filter Tool Window that displays by default in the upper-right sector of the Message Analyzer user interface.
On the toolbar of the Viewpoint Tool Window. After you apply any Viewpoint to a set of trace results, you can also apply a Viewpoint filter to further refine your analytical focus on specific messages.
In the Edit Color Rule dialog that displays when you click the New Color Rule item in the Color Rules drop-down list that appears on the toolbar of the Analysis Grid viewer. Typical configuration of a Color Rule includes specifying a Filter Expression from the centralized Library.
On the toolbar of the Find Message dialog that displays when you click the Find Message button on the toolbar of the Analysis Grid viewer.
To simplify the process of configuring and applying a View Filter, Message Analyzer enables you to right-click certain columns in the Analysis Grid viewer and automatically code the column value into a valid Filter Expression, which you can immediately apply to a set of trace results.
Creating Custom Filters
To create your own Filter Expressions, you will need to understand the Message Analyzer Filtering Language, as indicated earlier. When you create a custom Filter Expression, you have the option to save it in the centralized Filter Expression Library that is exposed in the locations described earlier in the Centralized Filter Expression Library subtopic. Also, before a Filter Expression that you create is actually applied to any data, Message Analyzer performs a simple verification check to ensure that it compiles as a valid expression; otherwise, you will need to reconfigure the expression or abandon it. To assist you in creating your own Filter Expressions, Message Analyzer provides the Filter IntelliSense service, which is an interactive and intelligent statement completion service that responds to the text that you enter in any Filter Expression text box.
To learn more about the Filter IntelliSense feature, see Filter IntelliSense Service.
To learn more about Fast Filters and WFP Layer Set filters, see the Microsoft-PEF-WFP-MessageProvider topic.
To learn more about Fast Filter Groups and System Network Adapter Group filters, see the PEF-NDIS Fast Filters and Using the Advanced Settings - Microsoft-PEF-NDIS-PacketCapture Dialog topics.
To learn more about HostName and Port filters, see the Microsoft-PEF-WebProxy Provider topic.
To learn more about Keyword event and error Level filters, see the System ETW Provider Configuration Settings topic.
To learn more about NDIS stack, Hyper-V-Switch extension layer, host adapter, and other special filters, see the Capturing Data Remotely and Using the Advanced Settings - Microsoft-Windows-NDIS-PacketCapture Dialog topics.
When analyzing data that you have either captured live on the network, loaded into Message Analyzer, or retrieved from a device such as a Bluetooth, you have the option to apply various types of filters to manipulate the way data is presented for analysis purposes. For example, you could apply various View Filter, Quick Filter, Color Rule, Column Filter, and Grouping configurations to a set of trace results, to name a few. In addition, you might use the Pattern Match capability to detect message patterns across a set of trace results.
To review summary descriptions of the analysis tools that are available in Message Analyzer, see Analyzing Message Data. For further details about the tools mentioned in this section, see More Information at the end of this section.
Data Analysis Feature Highlights
Some highlights of the options you have for manipulating data are included here in the following features.
Viewpoints — you have the option to apply a Viewpoint to enhance your data analysis and troubleshooting perspectives. When a Viewpoint is applied, you can examine network traffic from the perspective of a protocol because all messages above the “viewpoint” protocol are temporarily removed from display. This feature is advantageous when you have higher-layer traffic that obscures the underlying messages that you want to troubleshoot.
For example, you might apply a TCP Viewpoint to display TCP messages at top-level for diagnostic purposes. You might even select one of the TCP View Layouts to expose additional data field values that are particularly important to your analytical proceedings. You can do the same thing with an HTTP Viewpoint and the HTTP diagnosis layout. To apply a Viewpoint, click the Viewpoint drop-down list on the toolbar of the Viewpoint Tool Window that appears by default in the lower-right sector of the Message Analyzer user interface.
Grouping — you have the option to organize trace results data into Groups that expose messages in a nested Group configuration that you can specify by executing the Group command that appears when you right-click an Analysis Grid viewer column. You can also utilize the Grouping viewer to select built-in Grouping View Layouts that organize data into unique Group configurations that are designed to create a specific analytical focus, where you can summarize and expose target data across a high volume of messages. The Grouping viewer also provides different modes of interaction with the Analysis Grid viewer, which includes the Selection and Filtering modes.
Analysis Grid View Layouts — you have the option to apply built-in View Layouts that contain an arrangement of data columns that are designed to assist you in data analysis and troubleshooting processes, for example the File Sharing SMB/SMB2, Network Conversation Tree with Process ID, and TCP diagnosis layouts. You can select a View Layout from the Layout drop-down list on the toolbar of the Analysis Grid viewer in an Analysis Session.
Field Chooser — for any given set of data that displays in the Analysis Grid viewer, you should be aware that there are many more columns of data that you can add to the grid beyond the default column layout, to expose the values of message fields that could be critical to troubleshooting processes. You simply find the protocol or module of interest in the Field Chooser Tool Window, expand the protocol node, and navigate to the data field you want to add to the Analysis Grid viewer column layout. After you double-click the field name, the field is added to the Analysis Grid viewer as a new column, provided that the grid is in focus; at which time, data should display in the new field for the particular protocol or module of interest, unless the field is not used or contains no data.
If the Grouping Viewer is in focus, then double-clicking a field name in Field Chooser window will add a new nested Group to the current Grouping View Layout configuration. Note that you can also right-click any field in Field Chooser window and choose the Add as Column or Add as Grouping context menu command to add a column to the Analysis Grid viewer or a new nested Group to the Grouping viewer, respectively.
To access the Field Chooser window, if it is not already displayed, click the Add Columns button on the toolbar of the Analysis Grid viewer. If the Grouping viewer is in focus, you can access the Field Chooser by clicking the Add Groupings button on the toolbar of the Grouping viewer. The Field Chooser also appears in the Windows submenu of the global Message Analyzer Tools menu.
View Layouts and column layouts are different terms that essentially describe the same feature or function in Message Analyzer. Also note that the Grouping viewer has its own set of view Layouts that are independent of view Layouts for the Analysis Grid viewer.
Pattern Matching — provides a pattern matching capability that can identify sequential message patterns in a group of messages, for example virus signatures, processes in a faulty state that form a specific pattern, and other patterns such as request/response pairs. You can match message sequences by executing user-designed or built-in Pattern expressions that are provided with the Pattern Match viewer.
Filtering columns — you can apply a Column Filter to any Analysis Grid viewer column, to filter your trace results according to search text that you specify for a column. You can also do the same for columns in the Details Tool Window, to filter for specific message field names or other data values in the Details windows. Be aware that Column Filters search only on data that displays in top-level parent nodes; child nodes will not be included in the search unless you first expand them.
Aliases — if you have any data fields that are difficult to work with, due to their cryptic or complex naming, Message Analyzer enables you to convert their data values to a more user-friendly name for ease of recognition. To configure an Alias, right-click a field value that you want to convert and then select the Create Alias for ‘ <columnName> ’… context menu item. This action causes the Alias Editor to display, from where you can configure a new Alias.
Unions— if you have multiple data sources that relate to a common environment or service from which you have run traces or generated logs, it is not uncommon for the different data sources to specify different names for fields that have identical meaning and value types. If this occurs, you can create a Union to correlate the field values into a single, newly-named entity that you specify to reflect those values. This makes it easier to locate and analyze data in an interlaced set of messages from multiple sources.
Other Data Analysis Features
Other techniques that you can use to analyze data consist of the following:
Tool Windows — you can display additional Tool Windows to dramatically enhance the scope of analysis capabilities. These tools are accessible from the Windows submenu of the global Message Analyzer Tools menu.
Finding messages — you can use the Find Message feature to locate individual messages. The Find command is designated by the Find binoculars icon in the Find Message window that displays when you click the Find Messages button on the toolbar of the Analysis Grid viewer. This command enables you to locate the next message that matches a specified Filter Expression, while still retaining visibility and context of all the messages in the original trace results. You can select either a built-in Filter Expression from the Library drop-down list in the Find Message window, or you can manually configure one, such as contains “bing” (or some other string), or #MessageNumber==messagenumber, to locate the next message that contains the value that you specify.
By using this command, you can dramatically impact your data analysis experience. Although Message Analyzer already provides the View Filter capability that works similarly, the disadvantage of a View Filter is that all messages surrounding the target message are hidden after View Filter application, unless they match the filter criteria. However, in many cases the context of the surrounding messages is key to the analysis. When this is the case, it might be better to employ a Find filter. A Find filter highlights the next top-level message that matches the filtering criteria; however, note that the match might be to a message that is within the message stack of the highlighted top-level message, also known in this documentation as the origins tree.
You can also use the Go To Message feature to locate a specified message by its number across one or more data sources or sessions.
Sorting — you can sort data columns in the Analysis Grid viewer in ascending or descending order, to expose values or trends that can identify potential issues. For example, you can sort the DiagnosisTypes column of the Analysis Grid to bubble up all diagnostic messages for quick analysis.
Applying time shift values — Message Analyzer provides a Shift Time dialog that enables you to apply a specific incremental time shift value to a message collection from a selected data source, when you know beforehand that a time shift is required. You can also specify a time shift for a particular message when you discover through analysis that a shift is required. Applying a time shift to a selected message then causes a recalculation of time stamps for all messages in a selected data source.
You can use this feature to synchronize multiple traces that you load into Message Analyzer, for example to adjust for machine skew or time zone changes across traces. You might also want to simply match the Timestamp of one message loaded from a particular data source to that of another message loaded from a different data source.
Adding bookmarks and comments — you can add Bookmarks and Comments for annotation purposes to coordinate data analysis with other team members.
To learn more about the details of working with filters and other data manipulation features for analysis, see the following topics:
Applying and Managing Filters
Applying a Time Filter to Session Results
Using the Find Message Feature
Using the Go To Message Feature
Filtering Column Data
Using and Managing Color Rules
Applying and Managing Viewpoints
Pattern Match Viewer
Using the Analysis Grid Group Feature
Applying and Managing Analysis Grid View Layouts
Using the Field Chooser
Using and Managing Message Analyzer Aliases
Configuring and Managing Message Analyzer Unions
Setting Time Shifts
After you have performed analysis of your message data, you have the option to save it in the Message Analyzer native file format, as described in Saving Message Data. Thereafter, if you want to work further with the data or share it with others, you can quickly load the data back into Message Analyzer through a Data Retrieval Session, or by using the Open dialog, which is accessible from the global Message Analyzer File menu or on the global Message Analyzer toolbar.
The figure that follows illustrates the Save/Export Session dialog, in which you can choose the messages you want to save. You have the option to save all messages that you captured in a Live Trace Session or loaded from a Data Retrieval Session, filtered messages only, or you can select specific messages to save.
Figure 6: Message Analyzer Save/Export Session dialog
Note that there are two different ways to save specifically selected messages, as follows:
Use the Windows Save As dialog — highlight one or more messages in the Analysis Grid viewer, right-click the group of messages, and then select the Save Selected Messages… context menu command to display the Windows Save As dialog. After you specify a name for the session messages you are saving, click the Save button to save the session in the native .matp file format only.
Use the Message Analyzer Save As dialog options — highlight one or more messages in the Analysis Grid viewer and then click Save As in the File menu to display the Save As dialog. When you use this dialog to save data, you can specify additional save options with the use of three radio buttons under Step 1 of the dialog, which includes the following:
All Messages ()
Filtered Messages for <viewerName> view ()
Selected Messages for <sessionName> session ()
To save selected messages only, use the third option, which parenthetically indicates the number of messages that you highlighted in the in-focus data viewer. Thereafter, click the Save As button in Step 2 of the dialog to open the Windows Save As dialog, from where you can navigate to an appropriate directory location for saving the data in the native Message Analyzer .matp file format. To export the selected messages to a .cap file, click the Export button in Step 2 of the dialog to display the Windows Save As dialog.
If you have a session configuration that consists of an aggregation of data from multiple sources that you have analyzed, Message Analyzer enables you to save your results to a single file in the default .matp format. Note then when you export your data as a .cap file, it will be compatible with the Microsoft Network Monitor tool and other applications, with certain exceptions that are described in Compatibility with Exported CAP Files.
To learn more about saving Message Analyzer data, see Saving Message Data.