Network Monitor 3.3 RWS Parser Basics, Part 1: Introduction to RWS Protocol Analysis

Published: February 2010

Author:
Jim Harrison - Program Manager

Reviewers:
Yuri Diogenes - Sr Security Support Escalation Engineer
Michael Hawker - Program Manager
Rachel Aldam - Technical Writer

Editors:
David Strausberg - Technical Writer
Meir Feinberg- Technical Writer

Introduction

In my previous article on Forefront TMG Client basics, I described the history and basic mechanics behind the Forefront TMG Client. In this article, I’d like to introduce the Network Monitor 3.3 functionality and help you understand how it and the RWS parser can be used to help you resolve Forefront TMG Client issues in your own network.

There are three points that make it difficult to troubleshoot Forefront TMG Client traffic:

1. Winsock usage by the application under test. In most cases, certain assumptions can be made regarding the way a client would behave as a Winsock consumer. The more often you chase application and Forefront TMG Client issues, the more you will realize that because Winsock is a very flexible API, there are many ways to accomplish the goal of transferring data on the network. Likewise, more than a few Customer Support Services (CSS) cases were caused by developers who made assumptions about the client’s network structure and the result of these assumptions caused many problems when the client found itself operating in a Forefront TMG Client environment.

2. The random nature of the Forefront TMG Client application ports. As explained in the FWC 101 article, the ports that are used to pass application traffic from the Forefront TMG Client host through Forefront TMG to the remote server are not assigned until the application makes the relevant Winsock calls. Because the Forefront TMG Client and firewall communicate these ports to each other as part of the RWS protocol, it’s a relatively simple matter to parse these and assemble them into a capture summary...which brings us to point #3:

3. The Forefront TMG Client control channel may be encrypted. While this state is not guaranteed, it is how the Forefront TMG Client and firewall behave by default when user authentication is required by Forefront TMG. Because Forefront TMG Client is an important part of user authentication for non-HTTP clients, it’s very likely that you are using at least one authenticated access rule and that’s all it takes for Forefront TMG Client and Forefront TMG to encrypt the control channel. I’ll show you how to disable Forefront TMG Client encryption so that you can use network captures to resolve Forefront TMG Client problems. I’ll also show you how to re-enable encryption.

Remote Winsock (RWS) and Winsock Proxy (WSP) protocols

The FWC 101 article provided a functional example of how the Forefront TMG Client and Forefront TMG communicate to provide traffic flow for Winsock applications. What follows here is a functional description of the RWS protocol as used by the Forefront TMG Client. With some exceptions, this behavior is functionally identical to the way the FWC and ISA Server communicate.

RWS messages

As with any protocol, the data format that is used when the Forefront TMG Client and Forefront TMG communicate on the control channel follows a specific structure. Each message includes some information that is common to all messages, other data that is shared between related messagesin addition to data that is unique to that message alone. The primary data point in all RWS messages is the OpCode field.

- RWS: Channel setup response to ftp.exe (ISA Server compatible), authentication to CONTOSO\Forefront TMG01$ is required; encryption not required
 - RwstPacket: Channel setup response to ftp.exe (ISA Server compatible), authentication to CONTOSO\Forefront TMG01$ is required; encryption not required
   NullChar: 0 (0x0)
   ProtoSig: RWS
   PktLen: 301 (0x12D)
   Reserved1: 0 (0x0)
   Flags: 0 (0x0)
   Reserved2: 0 (0x0)
   OpCode: Channel setup
  + RwsMessage: response to ftp.exe (ISA Server compatible), authentication to CONTOSO\Forefront TMG01$ is required; encryption not required

The value in this field determines the context and content of the RWS message and conversation state.

The following tables provide a summary of and basic relationship between RWS messages. Items in the Precedes and Follows column fields that are italicized indicate that the RWS opcode may follow or precede those messages. This will become clearer as we continue through the application examples later. Except for the “v12” and Extended data messages, these messages are shared with ISA Server and the FWC.

RWS OpCodes that include WSP in the Precedes column indicate that these OpCodes are the last RWS message in a process that defines an application data channel.

Request OpCodes

Request messages are sent from the Forefront TMG Client to Forefront TMG. They are always the result of an application issuing a Winsock call or the Forefront TMG Client requesting action by or information from Forefront TMG.

OpCode	Message	Purpose	Follows	Precedes
0	Logon	Provides credentials using GssAPI	Channel setup response	Logon continue, Logon OK, Logon deny, Extended Data
7	Listen	Configures a bound socket to accept incoming traffic	Bind Reply v12	WSP
13	Get host by name	Requests an IP address for the specified name	Extended Data	Host Entry
14	Get host by address	Requests a name for the specified IP address	Extended Data	Host Entry
16	Close	Closes a bound, unconnected socket at Forefront TMG	Connect response v12, Listen reply, Accept Mapping v12	None
23	Set socket option	Instructs Forefront TMG to set the specified Winsock socket option	Connect v12 reply, Bind Reply v12	WSP
24	Get Server IPs	Requests Forefront TMG computer external IP addresses	Extended data	Bind v12
29	Ping	RWS control channel connectivity test	Extended Data	Ping response
32	Channel setup	Establish baseline data for RWS control channel	TCP connection to Forefront TMG on port 1745	Extended Data
33	Connect v12	Requests a connection to the specified IP address and port	Extended Data	Connect Reply v12
35	Bind v12	Instructs Forefront TMG to create a binding on the specified IP address and port	Extended Data	Listen, Connect v12
37	Mapping Request v12	Indicates an implicit bind action; initiated by a Winsock sendto() call	Extended Data	Add Mapping v12
41	Extended Data	Provides additional details about the Winsock application and Forefront TMG Client environment	Channel setup, Logon OK	Winsock commands

Reply OpCodes

Reply messages are sent in response to request messages, and express a success or failure state for the previous request. In either case, relevant clarifying data is included.

OpCode	Message	Purpose	Follows	Precedes
1	Error	Indicates that the previous request failed and provides error details	Any failed request	Retry
8	Listen reply	Indicates a success state and provides data for the previous request	Listen	Add Mapping v12
15	Host entry	Provides IP addresses and aliases to the previous request	Get host by name, Get host by address	Connect v12
20	Logon continue	Indicates that the Forefront TMG Client should continue sending credentials data	Logon	Logon
21	Logon OK	Indicates successful authentication	Logon, Logon continue	Extended Data
22	Logon deny	Indicates failed authentication	Logon, Logon continue	None
29	Ping	RWS control channel connectivity response	Ping request	None
32	Channel setup	Indicates criteria required by Forefront TMG to continue the RWS conversation	Channel setup request	Logon, Extended Data
34	Connect Reply v12	Indicates successful connection to the requested destination	Extended Data	WSP, Set Socket option
36	Bind Reply v12	Indicates successful Bind v12 action and provides some bind data	Extended Data	Listen

Notification OpCodes

Notification messages differ from reply messages because although they are sent relative to a previous Listen request, they are not the direct result of that action. Instead, they indicate that Forefront TMG needs to send traffic it has received on a listening socket and Forefront TMG Client should prepare to accept data on a connection negotiated in a previous Bind/Listen sequence.

OpCode	Message	Purpose	Follows	Precedes
38	Add Mapping v12	Instructs Forefront TMG Client to create a bound socket in preparation to send data	Extended data	Acknowledge Mapping v12
40	Remove Mapping v12	Instructs Forefront TMG Client to remove a previously defined mapping	Acknowledge Mapping v12	Close

Acknowledgement OpCodes

Acknowledgement messages are sent by the Forefront TMG Client to Forefront TMG to acknowledge receipt of the Mapping v12 message. These differ from request/reply messages in that they are only seen in the context of notification messages.

OpCode	Message	Purpose	Follows	Precedes
39	Acknowledge Mapping v12	Acknowledges the previous message	Add Mapping v12	WSP

ISA Server or FWC OpCodes

These messages are unique to the scenarios where the Forefront TMG Client is communicating to ISA Server or a Firewall Client is communicating to Forefront TMG or ISA Server (RWS v12 messages cannot be used).

OpCode	Message	Purpose	Follows	Precedes
3	Connect reply	Same as Connect Reply v12	New Connect	WSP
4	TCP bind	Same as Bind v12 for TCP only	Channel setup reply, Logon OK	Bind reply
5	UDP bind	Same as Bind v12 for UDP only	Channel setup reply, Logon OK	Bind reply
6	Bind reply	Reply to a TCP or UDP Bind	TCP Bind, UDP Bind	Listen
9	New mapping	Same as Add Mapping v12	Listen, Mapping request	Mapping Received
10	Mapping received	Same as Acknowledge Mapping v12	New Mapping	WSP
11	Mapping request	Same as Mapping Request v12	Channel setup reply, Logon OK	New Mapping
12	Remove mapping	Same as Remove Mapping v12	Mapping Received	None
30	New Connect	Same as Connect v12	Channel setup reply, Logon OK	Connect reply

Proxy Server 2 or WSP Client OpCodes

These messages are unique to communication between the Winsock Proxy Client used for Proxy Server. These are not supported by Forefront TMG.

OpCode	Message	Purpose
2	Connect	Same as New Connect and Connect v12
25 - 28	IPX Protocol Messages

Unused OpCodes

OpCode	Message	Purpose
17	Get serv by name	Requests data relevant to a service name
18	Get serv by port	Requests data relevant to a service port
19	Serv entry	Provides data related to the preceding Get serv by name or port
31	Never used

Initial RWS flow

The diagram in the figure below depicts the RWS process that must occur before any other RWS exchanges can take place.

In the Network Monitor frame summary window, this process will appear as shown in the following table:

Group	Source	Dest	Frame Summary
Connect to TCP:1745	Forefront TMG Client	Forefront TMG	TCP:Flags=......S., SrcPort=2579, DstPort=1745, PayloadLen=0, Seq=4290377438, Ack=0, Win=65535 ( ) = 65535
	Forefront TMG	Forefront TMG Client	TCP:Flags=...A..S., SrcPort=1745, DstPort=2579, PayloadLen=0, Seq=308431385, Ack=4290377439, Win=16384 ( Scale factor not supported ) = 16384
	Forefront TMG Client	Forefront TMG	TCP:Flags=...A...., SrcPort=2579, DstPort=1745, PayloadLen=0, Seq=4290377439, Ack=308431386, Win=65535 (scale factor 0x0) = 65535
Channel Setup	Forefront TMG Client	Forefront TMG	RWS:Channel setup request (Forefront TMG compatible) for iexplore.exe as username on computername running Windows XP x32
	Forefront TMG	Forefront TMG Client	RWS:Channel setup response to iexplore.exe (Forefront TMG Server compatible), authentication to CONTOSO\Forefront TMG01$ is required; encryption is required
Authentication	Forefront TMG Client	Forefront TMG	RWS:Logon (needs reassembly)
	Forefront TMG	Forefront TMG Client	TCP:Flags=...A...., SrcPort=1745, DstPort=2579, PayloadLen=0, Seq=308431685, Ack=4290379111, Win=65535 (scale factor 0x0) = 65535
	Forefront TMG	Forefront TMG Client	RWS:Logon continue
	Forefront TMG Client	Forefront TMG	RWS:Logon (needs reassembly)
	Forefront TMG Client	Forefront TMG	TCP:Flags=...A...., SrcPort=2579, DstPort=1745, PayloadLen=0, Seq=4290379878, Ack=308432129, Win=64792 (scale factor 0x0) = 64792
	Forefront TMG	Forefront TMG Client	RWS:Logon OK

The critical aspects of this process are whether Forefront TMG requires the client to authenticate or encrypt the RWS conversation that follows the logon process. These requirements are stipulated in the RWS Channel setup response message:

- RWS: Channel setup response to iexplore.exe (Forefront TMG compatible), authentication to CONTOSO\Forefront TMG01$ is required; encryption is required
 - RwstPacket: Channel setup response to iexplore.exe (Forefront TMG compatible), authentication to CONTOSO\Forefront TMG01$ is required; encryption is required
   NullChar: 0 (0x0)
   ProtoSig: RWS
   PktLen: 299 (0x12B)
   Reserved1: 0 (0x0)
   Flags: 0 (0x0)
   Reserved2: 0 (0x0)
   OpCode: Channel setup
  - RwsMessage: response to iexplore.exe (Forefront TMG compatible), authentication to CONTOSO\Forefront TMG01$ is required; encryption is required
  - SetupData: response to iexplore.exe (Forefront TMG compatible), authentication to CONTOSO\Forefront TMG01$ is required; encryption is required
    Padding: Binary Large Object (18 Bytes)
    MinVersion: Forefront TMG compatible
    MaxVersion: Forefront TMG compatible
    Authentication: required
    Reserved: 0 (0x0)
   - SetupFlags: KeepSession: False; RouteMode: True; ServerEncrypt: True; ClientEncrypt: True
    ServerKeepOldSession: (...............................0) Keep previous session for this application
    ClientRouteMode:    (..............................1.) Route mode desired
    ServerEncryption:   (.............................1..) Encryption required
    ClientEncryption:   (............................1...) Encryption enabled
    FlagsUnused:      (....000000000000000000000000....)
    dwReserved: 0 (0x0)
   + DiagBuf: 
    Padding: Binary Large Object (178 Bytes)
    FirewallContext: CONTOSO\Forefront TMG01$

RWS authentication is performed using:

• Forefront TMG Client or Forefront TMG in a Workgroup: Negotiate(NTLM)

• Forefront TMG Client and Forefront TMG in a Domain (trust): Negotiate(Kerberos)

When NTLM is used, the Logon continue message will contain an NTLM Negotiate message. When Kerberos is used, the Logon continue message will contain a Kerberos ticket for the Forefront TMG computer to satisfy mutual authentication.

If the credentials supplied by Forefront TMG Client are unacceptable to Forefront TMG for any reason, Forefront TMG will respond with Logon deny and close the control channel connection.

If Forefront TMG requires encryption and Forefront TMG Client were to send unencrypted RWS messages, Forefront TMG would simply reset the control channel connection with the Forefront TMG Client.

FWC / Forefront TMG Client Differences

There are some differences worth noting about the RWS protocol that are dependent on the combination of FWC/Forefront TMG Client and firewall.

ISA Server 2000 / FWC

• The RWS control channel uses UDP by default, although TCP was an option.

• The RWS control channel did not include encryption, even if TCP was used.

• The FWC does not support encryption, regardless of the firewall product.

ISA Server 2004 and 2006 / FWC

• The RWS control channel uses TCP by default, although UDP is an option.

• The RWS control channel uses encryption when authentication is required, although this can be disabled (see Tools and Techniques).

• 64-bit compatible FWC was produced separately.

Forefront TMG / Forefront TMG Client

• RWS protocol introduces “v12” versions of several messages, primarily in support of IPv6.

• The RWS control channel no longer supports UDP.

• The RWS control channel still uses encryption when authentication is required, but disabling it only requires action in Forefront TMG COM (see Tools and Techniques).

• All x64 updates from the ISA 2006 FWC are incorporated into the Forefront TMG Client.

Remote Winsock (RWS) and Winsock Proxy (WSP) Parser Properties

Two protocols are used by Forefront TMG Client and the FWC:

• RWS: the control channel where Winsock calls are translated by the Forefront TMG Client to Forefront TMG.

• WSP: the application abstraction protocol negotiated between the Forefront TMG Client and Forefront TMG.

The RWS protocol always occurs on port 1745, so identifying it in a protocol parser is pretty simple. Of course, it’s always possible that another application may be using this port on the network, but they do so in violation of IANA port registrations because RWS is registered with IANA as “remote-winsock” for both UDP and TCP port 1745. The RWS parser will determine if the packet is a valid RWS message and if it is not valid consume the remaining bytes in the packet to avoid inappropriate parsing by any parsers that might follow from within the TCP parser.

The WSP protocol represents the application traffic that occurs on the transport (TCP or UDP) and port negotiated between the Forefront TMG Client and Forefront TMG, so it will be observed only in the context of RWS traffic that preceded it.

Also, because these ports are assigned using Winsock dynamic port assignment, the port usage for this traffic will depend on the operating system where Forefront TMG and Forefront TMG Client are operating. As noted in MSKB 929851, these ports are assigned from different ranges, depending on the Windows version in use. For Windows 2003 and earlier, these are taken from the range 1025-5000 and for Vista and later, from the range 49152-65535. Thus, the RWS parser must pass data back to the TCP and UDP parsers so that they can recognize instances of WSP traffic and pass the traffic to the WSP parser. Luckily, the Network Monitor team foresaw this need and designed the parser engine so that parsers could communicate with each other.

In any protocol parser, properties are declared and defined in support of protocol state description and management. These properties can be used in capture and display filters as well as any Network Monitor experts. Quite a number of properties are defined for use within the RWS parser to communicate to itself between frames as well as to communicate with the parent (TCP and UDP) protocols. Some of these properties are passed to the WSP parser to help you associate the WSP conversation with the RWS conversation that created them. The primary properties we defined to assist you with troubleshooting Forefront TMG Client-related issues are:

Property Name	Description
Conversation.RwsApp	the name of the Winsock application, such as “ftp.exe” Note: This field is limited to16 characters
Conversation.RwsHost	the name of the computer where Forefront TMG Client is operating, such as “jimsbox”
Conversation.RwsId	a binary value derived from the RWS Client Connection Handle value
Conversation.RwsUser	the name of the account used to run the Winsock application

These properties are all defined in the RWS parser and passed to the WSP parser through global property sets. The Network Monitor 3.3 help discusses global properties in detail, so I won’t belabor the point here. The RWS and WSP parsers also include Property.RwsApp, Property.RwsHost, Property.RwsId and Property.RwsUser properties. I’ll describe how to use these when I discuss HTTP via Forefront TMG Client and DNS via Forefront TMG Client in part 2 and 3 of this article series.

All of the RWS parser properties are derived from the RWS messages exchanged between the Forefront TMG Client and Forefront TMG in the process of establishing a traffic path for the client application. Because the majority of this data is internal traffic management or serialized Winsock calls, much of the data in RWS frames may be new to you. Luckily, most problems tend to be simple things such as name resolution failure, attempting to connect an invalid destination IP or port or even normal problems such as an application server listener that isn’t listening. This means you generally won’t have to be concerned with the details of the Winsock calls or how the Forefront TMG Client forwards them to Forefront TMG.

Winsock Applications

In the FWC 101 article, I used an example of an FTP client connection because the FTP protocol is fairly well-understood in the networking community and it illustrates the most commonly-used Winsock calls that generate RWS messages (getaddrinfo(), connect(), bind(), listen(), send(), recv()).

In general, most client-side Winsock applications will fall into a predictable Winsock usage pattern. As a troubleshooter, it’s deviations from these patterns that you will be most interested in identifying. For instance, a Web browser that is not configured to behave as a Web proxy client will perform the following simplified Winsock call sequence:

1. Resolve the URL hostname to an IP address: getaddrinfo(“hostname”)

2. Connect to the remote server: connect(ip.add.re.ss:port)

3. Exchange data on that connection: send(), recv()

These actions translate to the following TGMC and Forefront TMG actions:

1. Resolve the URL hostname to an IP address

   1. Forefront TMG Client issues gethostbyname(“hostname”) request to Forefront TMG

   2. Forefront TMG resolves “hostname” to an IP address (may call Winsock to do this)

   3. Forefront TMG responds with Host Entry to Forefront TMG Client

2. Connect to the remote server

   1. Forefront TMG Client issues Connect v12 (ip.add.re.ss:port) request to Forefront TMG. This request also informs Forefront TMG which IP address and port the Forefront TMG Client will use for the application data connection

   2. Forefront TMG connects to ip.add.re.ss:port

   3. Forefront TMG sends Connect v12 response to Forefront TMG Client (includes Forefront TMG side of the application data connection if successful)

3. Send and receive traffic on that connection

   1. Forefront TMG Client connects to Forefront TMG end of the application data connection

   2. Forefront TMG Client routes outgoing application data through Forefront TMG using the application data connection

   3. Forefront TMG routes incoming data to Forefront TMG Client using the application data connection

   Note

   You may not see name lookups in the RWS conversation if the client is able to resolve the name using a local DNS service.

Other Winsock applications may exhibit more complex behavior, such as creating listeners for inbound data such as that seen with active FTP or peer-to-peer (P2P) applications or creating multiple outbound connections, such that as seen with may browsers or passive FTP, or perhaps any combination of these.

You can see how this process occurs in a network capture, but only if you use Network Monitor 3.3 and later. Network Monitor 3.3 is currently the only network analysis tool that understands the Forefront TMG Client protocols. You can get the latest Network Monitor 3.3 package and the latest parsers from the links in the References section.

When you are troubleshooting Forefront TMG Client problems, it’s often best to obtain multiple concurrent captures – one at the Forefront TMG Client client computer and one at the Forefront TMG computer. The purpose in doing so is to gather as much data relevant to the two computers as possible in one session. In many cases, a capture taken at Forefront TMG alone could miss a local name query from the client. Likewise, a capture taken at the client alone will not indicate a failed connection attempt by Forefront TMG.

Network Monitor 3.3 Basics

Network Monitor 3.3 improves on Network Monitor 2 as provided with Windows 2000 and Windows 2003 in many significant ways. Parsers are far less likely to cause Network Monitor to fail because of the “sandboxed” design inherent in the parser engine, display and capture filters can be defined much more easily (as well as being imported and exported) and much more. In terms of troubleshooting Forefront TMG Client-managed traffic, the Network Monitor 3.3 Parser Language (NPL) made it much easier to develop and test the RWS parser and in doing so, allowed the Forefront Edge CS team the ability to extend the Network Monitor traffic analysis capability for Forefront TMG Client troubleshooting.

Traffic Capture Methods

I have four general rules for gathering network traffic:

1. Use command-line tools whenever possible. The extra CPU and memory expense involved with displaying traffic as it is captured can be significant; especially on a host that is already resource-constrained, such as a heavily-tasked Forefront TMG computer. Network Monitor provides nmcap.exe for this very purpose. You and your customers will benefit greatly through your becoming familiar with this tool. In fact, ISABPA and TMGBPA both gather network captures using nmcap.exe.

2. Use no capture filters whenever possible. Of course there will be exceptions to this rule such as when you already know what protocols you’re chasing, but for the most part, it’s much more effective to gather all of the data extant “on the wire” and filter out what you don’t want to see and change your display filters as your traffic analysis progresses.

3. Capture the traffic where you can observe the most data at once. Just because the user complains doesn’t mean the interesting data is available at her computer or at Forefront TMG internal NIC. It may be seen only at the external NIC, which means capturing at the client will produce interesting data, but probably nothing that is useful in the current effort.

4. Try to limit the amount of time spent capturing. Network captures consume disk and memory and the larger they are; the longer it takes to parse them. This may mean that you coordinate the capture and repro efforts with the user, or that you perform the capture and repro yourself.

Display filters

Learning to use Network Monitor display filters is time well spent. The general syntax is clean and lends itself to simplifying display filters that can be applied to multiple scenarios. Most users find themselves operating in the context of the most basic aspects of the protocols they’re chasing; TCP, UDP, ICMP, HTTP, SMTP, etc. While this is a great starting point for drilling down to the problem area, what you’re missing out on when you limit your filtering skills to the base protocol are the finer points of filter mechanisms through the use of protocol properties. For example, a simple display filter trying to identify Web proxy traffic between a client at 10.10.10.10 and a Forefront TMG server at 10.10.10.1 might be defined as:

(ipv4.address==10.10.10.10 and ipv4.address==10.10.10.1) and http

Of course, this display filter excludes any TCP traffic that is not identified as HTTP as well as any DNS or WINS traffic sent by the client that may precede the TCP connection between the client and Forefront TMG. Consequently, if the problem you’re chasing is WPAD failure caused by a name lookup failure for “wpad”, this display filter will result in a Network Monitor packet summary display window that is missing this name query. Understanding the peripheral network processes that are involved with communications between two hosts is critical to solving a problem through network analysis.

Likewise, a simple filter that could help you troubleshoot Web Proxy Automatic discovery (WPAD) might be written as:

HTTP.Request.URI=="/wpad.dat"

If you apply this display filter to a WPAD capture, you’ll see the results indicated in the figure below:

While this display filter might show you all instances where a Web client sent a request for “/wpad.dat”, it won’t show you any response that the application may have received to this request. The HTTP protocol doesn’t specify that the server response must include any part of the original request in the response, so you can’t look for “/wpad.dat” there. You can however look for other data that will be delivered by Forefront TMG when it responds to these requests. If you capture this process and examine the response from Forefront TMG, you will see the following:

- Http: Response, HTTP/1.1, Status: Ok, URL: /wpad.dat 
  ProtocolVersion: HTTP/1.1
  StatusCode: 200, Ok
  Reason: OK
  Date: Tue, 28 Apr 2009 20:59:37 GMT
  Connection: close
 - ContentType: application/x-ns-proxy-autoconfig
   MediaType: application/x-ns-proxy-autoconfig
  Cache-Control: max-age=3000
  HeaderEnd: CRLF
 + payload: HttpContentType = application/x-ns-proxy-autoconfig

The Forefront TMG response to a request for the WPAD script is always delivered with a content-type of “application/x-ns-proxy-autoconfig”, making it possible for you to define a display filter that includes the WPAD response as well as the request:

HTTP.Request.URI=="/wpad.dat" or HTTP.Response.HeaderFields.ContentType.MediaType == " application/x-ns-proxy-autoconfig"

Applying this display filter will produce results similar to that shown in the figure below:

Bear in mind that the filters I’ve just described don’t include the full WPAD protocol, which includes DNS (or WINS) as well as HTTP. The good news is that display filters written for proxy scenarios will soon be part of Network Monitor. Until then, feel free to see what criteria you can identify to help you define your own filters.

Summary

I’ve outlined the RWS command set and discussed the mandatory initial RWS command sequence and useful RWS parser properties. In Part 2, I’ll provide examples of single-connection TCP using HTTP as the application protocol between Forefront TMG Client and Forefront TMG. In part 3, I’ll cover single-connection UDP traffic between Forefront TMG Client and Forefront TMG using DNS as the application protocol.

Tools / Techniques

I’ve compiled a set of useful tools and methodologies to make your RWS troubleshooting go just a tad smoother.

Control Channel Encryption

If you require authentication for even one access rule, ISA Server and Forefront TMG will require RWS encryption after a successful Logon cycle. While this helps to prevent anyone sniffing RWS conversations on the wire, it also prevents you doing the same thing. Needless to say, this makes it somewhat difficult to troubleshoot RWS conversations.

ISA Server 2004 and 2006

1. Open the ISA Server management console.

2. In the left pane, do the following:

   1. In Enterprise Edition, expand Arrays, and then <ArrayName>, in Standard Edition, expand <ArrayName>.

   2. Expand Configuration

   3. Select General.

3. In the details pane, click Define Firewall Client Settings.

4. In the Firewall Client Settings window, do the following:

   1. Select Allow non-encrypted Firewall client connections.

   2. Click Apply, and then click OK.

5. To save your changes, on the Apply Changes bar, click Apply.

FWC

This configuration can create some interesting behavior if you don’t disable it when your troubleshooting is finished. If the FWC has encryption disabled, but ISA Server or Forefront TMG requires it, your FWC applications will not work. To disable encryption for the Firewall Client, you must set some registry values. The commands to accomplish this are listed below.

1. Click Start, All Programs, Accessories, Command Prompt.

   Note

   For Windows Vista and later, right-click Command Prompt and select Run as Administrator.

2. To disable Control Channel encryption, in the command window, enter the following command:

   32-bit Windows: 
   reg add "HKLM\Software\Microsoft\Firewall Client 2004\Policies" /v WspEncryptControlChannel /t REG_DWORD /v 0 /f
   
   64-bit Windows: 
   reg add "HKLM\Software\Wow6432Node\Microsoft\Firewall Client 2004\Policies" /v WspEncryptControlChannel /t REG_DWORD /v 0 /f
   

   There is no need to restart the firewall or the firewall agent for these changes to take effect.

   Be sure to wait until the ISA array has synchronized the policy before starting your tests or confusion will certainly ensue.

3. When finished testing, re-enable Control Channel encryption with the following command:

   32-bit Windows: 
   reg delete "HKLM\Software\Microsoft\Firewall Client 2004\Policies" /v WspEncryptControlChannel /f
   
   64-bit Windows: 
   reg delete "HKLM\Software\Wow6432Node\Microsoft\Firewall Client 2004\Policies" /v WspEncryptControlChannel /f
   

Forefront TMG and Forefront TMG Client

Because the “Allow unencrypted Firewall client connections” also allowed UDP-based control channel connections and Forefront TMG no longer supports the UDP control channel, this control was provided only in the Forefront TMG scripting COM. To disable control channel encryption, you have to run a script against the array where you want this action to take effect.

VBScript

Option Explicit
Dim oRoot: Set oRoot = CreateObject("FPC.Root")
Dim oArray: Set oArray = Root.GetContainingArray
Dim oClntCfg: Set oClntCfg = oArray.FWClientConfigSettings
oClntCfg.EnableControlChannelEncryption = Not( oClntCfg.EnableControlChannelEncryption )
oArray.Save(true, true)

1. Save this script as c:\togglerwsencryption.vbs.

2. Run this script on the Forefront TMG computer from an elevated command window as cscript c:\togglerwsencryption.vbs.

JScript

var oRoot = new ActiveXObject("FPC.Root");
var oArray = oRoot.GetContainingArray();
var oClntCfg = oArray.FWClientConfigSettings;
oClntCfg.EnableControlChannelEncryption = !oClntCfg.EnableControlChannelEncryption;
oArray.Save(true, true);

1. Save this script as c:\togglerwsencryption.js.

2. Run this script on the Forefront TMG computer from an elevated command window as cscript c:\togglerwsencryption.js.

Be sure to wait until the Forefront TMG array has synchronized the policy before starting your tests or confusion will certainly ensue.

Unlike ISA Server and the FWC, there is no need to change anything at the client when Forefront TMG is configured this way.

To re-enable control channel encryption, simply re-run the script and wait for the policy to synchronize.

Network Monitor downloads

1. Network Monitor 3.3 (https://www.microsoft.com/downloads/details.aspx?FamilyID=983b941d-06cb-4658-b7f6-3088333d062f)

2. Latest Network Monitor 3.3 parser (https://nmparsers.codeplex.com/)

References

1. Firewall Client Basics: Introduction to the ISA Server Firewall Client and Forefront TMG Client

2. IANA Port Assignments (https://www.iana.org/assignments/port-numbers)

3. Microsoft Knowledgebase article 929851 Winsock Ephemeral Port assignments (https://support.microsoft.com/kb/929851/)

4. Microsoft Knowledgebase article 196271 Ephemeral Port Range Adjustment (https://support.microsoft.com/kb/196271/)

5. ISABlog on Client / Firewall compatibility (https://blogs.technet.com/isablog/archive/2009/11/03/forefront-tmg-client.aspx)

6. Network Monitor 3.3 RWS Parser Basics, Part 2: Observing single-connection HTTP traffic

7. Network Monitor 3.3 RWS Parser Basics, Part 3: Observing single-connection DNS traffic