Network Monitor 3.3 RWS Parser Basics, Part 1: Introduction to RWS Protocol Analysis
Published: February 2010
Author:
Jim Harrison - Program Manager
Reviewers:
Yuri Diogenes - Sr Security Support Escalation Engineer
Michael Hawker - Program Manager
Rachel Aldam - Technical Writer
Editors:
David Strausberg - Technical Writer
Meir Feinberg- Technical Writer
Introduction
In my previous article on Forefront TMG Client basics, I described the history and basic mechanics behind the Forefront TMG Client. In this article, I’d like to introduce the Network Monitor 3.3 functionality and help you understand how it and the RWS parser can be used to help you resolve Forefront TMG Client issues in your own network.
There are three points that make it difficult to troubleshoot Forefront TMG Client traffic:
Winsock usage by the application under test. In most cases, certain assumptions can be made regarding the way a client would behave as a Winsock consumer. The more often you chase application and Forefront TMG Client issues, the more you will realize that because Winsock is a very flexible API, there are many ways to accomplish the goal of transferring data on the network. Likewise, more than a few Customer Support Services (CSS) cases were caused by developers who made assumptions about the client’s network structure and the result of these assumptions caused many problems when the client found itself operating in a Forefront TMG Client environment.
The random nature of the Forefront TMG Client application ports. As explained in the FWC 101 article, the ports that are used to pass application traffic from the Forefront TMG Client host through Forefront TMG to the remote server are not assigned until the application makes the relevant Winsock calls. Because the Forefront TMG Client and firewall communicate these ports to each other as part of the RWS protocol, it’s a relatively simple matter to parse these and assemble them into a capture summary...which brings us to point #3:
The Forefront TMG Client control channel may be encrypted. While this state is not guaranteed, it is how the Forefront TMG Client and firewall behave by default when user authentication is required by Forefront TMG. Because Forefront TMG Client is an important part of user authentication for non-HTTP clients, it’s very likely that you are using at least one authenticated access rule and that’s all it takes for Forefront TMG Client and Forefront TMG to encrypt the control channel. I’ll show you how to disable Forefront TMG Client encryption so that you can use network captures to resolve Forefront TMG Client problems. I’ll also show you how to re-enable encryption.
Remote Winsock (RWS) and Winsock Proxy (WSP) protocols
The FWC 101 article provided a functional example of how the Forefront TMG Client and Forefront TMG communicate to provide traffic flow for Winsock applications. What follows here is a functional description of the RWS protocol as used by the Forefront TMG Client. With some exceptions, this behavior is functionally identical to the way the FWC and ISA Server communicate.
RWS messages
As with any protocol, the data format that is used when the Forefront TMG Client and Forefront TMG communicate on the control channel follows a specific structure. Each message includes some information that is common to all messages, other data that is shared between related messagesin addition to data that is unique to that message alone. The primary data point in all RWS messages is the OpCode field.
- RWS: Channel setup response to ftp.exe (ISA Server compatible), authentication to CONTOSO\Forefront TMG01$ is required; encryption not required
- RwstPacket: Channel setup response to ftp.exe (ISA Server compatible), authentication to CONTOSO\Forefront TMG01$ is required; encryption not required
NullChar: 0 (0x0)
ProtoSig: RWS
PktLen: 301 (0x12D)
Reserved1: 0 (0x0)
Flags: 0 (0x0)
Reserved2: 0 (0x0)
OpCode: Channel setup
+ RwsMessage: response to ftp.exe (ISA Server compatible), authentication to CONTOSO\Forefront TMG01$ is required; encryption not required
The value in this field determines the context and content of the RWS message and conversation state.
The following tables provide a summary of and basic relationship between RWS messages. Items in the Precedes and Follows column fields that are italicized indicate that the RWS opcode may follow or precede those messages. This will become clearer as we continue through the application examples later. Except for the “v12” and Extended data messages, these messages are shared with ISA Server and the FWC.
RWS OpCodes that include WSP in the Precedes column indicate that these OpCodes are the last RWS message in a process that defines an application data channel.
Request OpCodes
Request messages are sent from the Forefront TMG Client to Forefront TMG. They are always the result of an application issuing a Winsock call or the Forefront TMG Client requesting action by or information from Forefront TMG.
OpCode | Message | Purpose | Follows | Precedes |
---|---|---|---|---|
0 |
Logon |
Provides credentials using GssAPI |
Channel setup response |
Logon continue, Logon OK, Logon deny, Extended Data |
7 |
Listen |
Configures a bound socket to accept incoming traffic |
Bind Reply v12 |
WSP |
13 |
Get host by name |
Requests an IP address for the specified name |
Extended Data |
Host Entry |
14 |
Get host by address |
Requests a name for the specified IP address |
Extended Data |
Host Entry |
16 |
Close |
Closes a bound, unconnected socket at Forefront TMG |
Connect response v12, Listen reply, Accept Mapping v12 |
None |
23 |
Set socket option |
Instructs Forefront TMG to set the specified Winsock socket option |
Connect v12 reply, Bind Reply v12 |
WSP |
24 |
Get Server IPs |
Requests Forefront TMG computer external IP addresses |
Extended data |
Bind v12 |
29 |
Ping |
RWS control channel connectivity test |
Extended Data |
Ping response |
32 |
Channel setup |
Establish baseline data for RWS control channel |
TCP connection to Forefront TMG on port 1745 |
Extended Data |
33 |
Connect v12 |
Requests a connection to the specified IP address and port |
Extended Data |
Connect Reply v12 |
35 |
Bind v12 |
Instructs Forefront TMG to create a binding on the specified IP address and port |
Extended Data |
Listen, Connect v12 |
37 |
Mapping Request v12 |
Indicates an implicit bind action; initiated by a Winsock sendto() call |
Extended Data |
Add Mapping v12 |
41 |
Extended Data |
Provides additional details about the Winsock application and Forefront TMG Client environment |
Channel setup, Logon OK |
Winsock commands |
Reply OpCodes
Reply messages are sent in response to request messages, and express a success or failure state for the previous request. In either case, relevant clarifying data is included.
OpCode | Message | Purpose | Follows | Precedes |
---|---|---|---|---|
1 |
Error |
Indicates that the previous request failed and provides error details |
Any failed request |
Retry |
8 |
Listen reply |
Indicates a success state and provides data for the previous request |
Listen |
Add Mapping v12 |
15 |
Host entry |
Provides IP addresses and aliases to the previous request |
Get host by name, Get host by address |
Connect v12 |
20 |
Logon continue |
Indicates that the Forefront TMG Client should continue sending credentials data |
Logon |
Logon |
21 |
Logon OK |
Indicates successful authentication |
Logon, Logon continue |
Extended Data |
22 |
Logon deny |
Indicates failed authentication |
Logon, Logon continue |
None |
29 |
Ping |
RWS control channel connectivity response |
Ping request |
None |
32 |
Channel setup |
Indicates criteria required by Forefront TMG to continue the RWS conversation |
Channel setup request |
Logon, Extended Data |
34 |
Connect Reply v12 |
Indicates successful connection to the requested destination |
Extended Data |
WSP, Set Socket option |
36 |
Bind Reply v12 |
Indicates successful Bind v12 action and provides some bind data |
Extended Data |
Listen |
Notification OpCodes
Notification messages differ from reply messages because although they are sent relative to a previous Listen request, they are not the direct result of that action. Instead, they indicate that Forefront TMG needs to send traffic it has received on a listening socket and Forefront TMG Client should prepare to accept data on a connection negotiated in a previous Bind/Listen sequence.
OpCode | Message | Purpose | Follows | Precedes |
---|---|---|---|---|
38 |
Add Mapping v12 |
Instructs Forefront TMG Client to create a bound socket in preparation to send data |
Extended data |
Acknowledge Mapping v12 |
40 |
Remove Mapping v12 |
Instructs Forefront TMG Client to remove a previously defined mapping |
Acknowledge Mapping v12 |
Close |
Acknowledgement OpCodes
Acknowledgement messages are sent by the Forefront TMG Client to Forefront TMG to acknowledge receipt of the Mapping v12 message. These differ from request/reply messages in that they are only seen in the context of notification messages.
OpCode | Message | Purpose | Follows | Precedes |
---|---|---|---|---|
39 |
Acknowledge Mapping v12 |
Acknowledges the previous message |
Add Mapping v12 |
WSP |
ISA Server or FWC OpCodes
These messages are unique to the scenarios where the Forefront TMG Client is communicating to ISA Server or a Firewall Client is communicating to Forefront TMG or ISA Server (RWS v12 messages cannot be used).
OpCode | Message | Purpose | Follows | Precedes |
---|---|---|---|---|
3 |
Connect reply |
Same as Connect Reply v12 |
New Connect |
WSP |
4 |
TCP bind |
Same as Bind v12 for TCP only |
Channel setup reply, Logon OK |
Bind reply |
5 |
UDP bind |
Same as Bind v12 for UDP only |
Channel setup reply, Logon OK |
Bind reply |
6 |
Bind reply |
Reply to a TCP or UDP Bind |
TCP Bind, UDP Bind |
Listen |
9 |
New mapping |
Same as Add Mapping v12 |
Listen, Mapping request |
Mapping Received |
10 |
Mapping received |
Same as Acknowledge Mapping v12 |
New Mapping |
WSP |
11 |
Mapping request |
Same as Mapping Request v12 |
Channel setup reply, Logon OK |
New Mapping |
12 |
Remove mapping |
Same as Remove Mapping v12 |
Mapping Received |
None |
30 |
New Connect |
Same as Connect v12 |
Channel setup reply, Logon OK |
Connect reply |
Proxy Server 2 or WSP Client OpCodes
These messages are unique to communication between the Winsock Proxy Client used for Proxy Server. These are not supported by Forefront TMG.
OpCode | Message | Purpose |
---|---|---|
2 |
Connect |
Same as New Connect and Connect v12 |
25 - 28 |
IPX Protocol Messages |
Unused OpCodes
OpCode | Message | Purpose |
---|---|---|
17 |
Get serv by name |
Requests data relevant to a service name |
18 |
Get serv by port |
Requests data relevant to a service port |
19 |
Serv entry |
Provides data related to the preceding Get serv by name or port |
31 |
Never used |
Initial RWS flow
The diagram in the figure below depicts the RWS process that must occur before any other RWS exchanges can take place.
In the Network Monitor frame summary window, this process will appear as shown in the following table:
Group | Source | Dest | Frame Summary |
---|---|---|---|
Connect to TCP:1745
|
Forefront TMG Client |
Forefront TMG |
TCP:Flags=......S., SrcPort=2579, DstPort=1745, PayloadLen=0, Seq=4290377438, Ack=0, Win=65535 ( ) = 65535 |
Forefront TMG |
Forefront TMG Client |
TCP:Flags=...A..S., SrcPort=1745, DstPort=2579, PayloadLen=0, Seq=308431385, Ack=4290377439, Win=16384 ( Scale factor not supported ) = 16384 |
|
Forefront TMG Client |
Forefront TMG |
TCP:Flags=...A...., SrcPort=2579, DstPort=1745, PayloadLen=0, Seq=4290377439, Ack=308431386, Win=65535 (scale factor 0x0) = 65535 |
|
Channel Setup
|
Forefront TMG Client |
Forefront TMG |
RWS:Channel setup request (Forefront TMG compatible) for iexplore.exe as username on computername running Windows XP x32 |
Forefront TMG |
Forefront TMG Client |
RWS:Channel setup response to iexplore.exe (Forefront TMG Server compatible), authentication to CONTOSO\Forefront TMG01$ is required; encryption is required |
|
Authentication
|
Forefront TMG Client |
Forefront TMG |
RWS:Logon (needs reassembly) |
Forefront TMG |
Forefront TMG Client |
TCP:Flags=...A...., SrcPort=1745, DstPort=2579, PayloadLen=0, Seq=308431685, Ack=4290379111, Win=65535 (scale factor 0x0) = 65535 |
|
Forefront TMG |
Forefront TMG Client |
RWS:Logon continue |
|
Forefront TMG Client |
Forefront TMG |
RWS:Logon (needs reassembly) |
|
Forefront TMG Client |
Forefront TMG |
TCP:Flags=...A...., SrcPort=2579, DstPort=1745, PayloadLen=0, Seq=4290379878, Ack=308432129, Win=64792 (scale factor 0x0) = 64792 |
|
Forefront TMG |
Forefront TMG Client |
RWS:Logon OK |
Tip
You may have noticed that the Logon messages in the frame summary include “(needs reassembly)”. This indicates that the RWS parser has determined that the Logon message is actually larger than this packet. In order for you to examine this RWS message completely, you must use the Network Monitor Reassemble button. Clicking this button will cause Network Monitor to reparse the capture and create additional packets interspersed in the capture that represent all fragmented RWS messages as they would be seen “reassembled” in the computer’s network stack.
The critical aspects of this process are whether Forefront TMG requires the client to authenticate or encrypt the RWS conversation that follows the logon process. These requirements are stipulated in the RWS Channel setup response message:
- RWS: Channel setup response to iexplore.exe (Forefront TMG compatible), authentication to CONTOSO\Forefront TMG01$ is required; encryption is required
- RwstPacket: Channel setup response to iexplore.exe (Forefront TMG compatible), authentication to CONTOSO\Forefront TMG01$ is required; encryption is required
NullChar: 0 (0x0)
ProtoSig: RWS
PktLen: 299 (0x12B)
Reserved1: 0 (0x0)
Flags: 0 (0x0)
Reserved2: 0 (0x0)
OpCode: Channel setup
- RwsMessage: response to iexplore.exe (Forefront TMG compatible), authentication to CONTOSO\Forefront TMG01$ is required; encryption is required
- SetupData: response to iexplore.exe (Forefront TMG compatible), authentication to CONTOSO\Forefront TMG01$ is required; encryption is required
Padding: Binary Large Object (18 Bytes)
MinVersion: Forefront TMG compatible
MaxVersion: Forefront TMG compatible
Authentication: required
Reserved: 0 (0x0)
- SetupFlags: KeepSession: False; RouteMode: True; ServerEncrypt: True; ClientEncrypt: True
ServerKeepOldSession: (...............................0) Keep previous session for this application
ClientRouteMode: (..............................1.) Route mode desired
ServerEncryption: (.............................1..) Encryption required
ClientEncryption: (............................1...) Encryption enabled
FlagsUnused: (....000000000000000000000000....)
dwReserved: 0 (0x0)
+ DiagBuf:
Padding: Binary Large Object (178 Bytes)
FirewallContext: CONTOSO\Forefront TMG01$
RWS authentication is performed using:
Forefront TMG Client or Forefront TMG in a Workgroup: Negotiate(NTLM)
Forefront TMG Client and Forefront TMG in a Domain (trust): Negotiate(Kerberos)
Note
If the Forefront TMG computer is specified to Forefront TMG Client as an IP address rather than a qualified or unqualified name, Kerberos authentication cannot be used.
When NTLM is used, the Logon continue message will contain an NTLM Negotiate message. When Kerberos is used, the Logon continue message will contain a Kerberos ticket for the Forefront TMG computer to satisfy mutual authentication.
If the credentials supplied by Forefront TMG Client are unacceptable to Forefront TMG for any reason, Forefront TMG will respond with Logon deny and close the control channel connection.
If Forefront TMG requires encryption and Forefront TMG Client were to send unencrypted RWS messages, Forefront TMG would simply reset the control channel connection with the Forefront TMG Client.
Note
The remainder of the RWS communication process is entirely dependent on the Winsock application. I’ll discuss those in parts 2 and 3.
FWC / Forefront TMG Client Differences
There are some differences worth noting about the RWS protocol that are dependent on the combination of FWC/Forefront TMG Client and firewall.
ISA Server 2000 / FWC
The RWS control channel uses UDP by default, although TCP was an option.
The RWS control channel did not include encryption, even if TCP was used.
The FWC does not support encryption, regardless of the firewall product.
ISA Server 2004 and 2006 / FWC
The RWS control channel uses TCP by default, although UDP is an option.
The RWS control channel uses encryption when authentication is required, although this can be disabled (see Tools and Techniques).
64-bit compatible FWC was produced separately.
Forefront TMG / Forefront TMG Client
RWS protocol introduces “v12” versions of several messages, primarily in support of IPv6.
The RWS control channel no longer supports UDP.
The RWS control channel still uses encryption when authentication is required, but disabling it only requires action in Forefront TMG COM (see Tools and Techniques).
All x64 updates from the ISA 2006 FWC are incorporated into the Forefront TMG Client.
Remote Winsock (RWS) and Winsock Proxy (WSP) Parser Properties
Two protocols are used by Forefront TMG Client and the FWC:
RWS: the control channel where Winsock calls are translated by the Forefront TMG Client to Forefront TMG.
WSP: the application abstraction protocol negotiated between the Forefront TMG Client and Forefront TMG.
The RWS protocol always occurs on port 1745, so identifying it in a protocol parser is pretty simple. Of course, it’s always possible that another application may be using this port on the network, but they do so in violation of IANA port registrations because RWS is registered with IANA as “remote-winsock” for both UDP and TCP port 1745. The RWS parser will determine if the packet is a valid RWS message and if it is not valid consume the remaining bytes in the packet to avoid inappropriate parsing by any parsers that might follow from within the TCP parser.
The WSP protocol represents the application traffic that occurs on the transport (TCP or UDP) and port negotiated between the Forefront TMG Client and Forefront TMG, so it will be observed only in the context of RWS traffic that preceded it.
Note
Because WSP conversations are described by the preceding RWS conversation, if the capture you’re analyzing doesn’t include relevant parts of the RWS conversation, the WSP parser won’t recognize any WSP traffic that may exist.
Also, because these ports are assigned using Winsock dynamic port assignment, the port usage for this traffic will depend on the operating system where Forefront TMG and Forefront TMG Client are operating. As noted in MSKB 929851, these ports are assigned from different ranges, depending on the Windows version in use. For Windows 2003 and earlier, these are taken from the range 1025-5000 and for Vista and later, from the range 49152-65535. Thus, the RWS parser must pass data back to the TCP and UDP parsers so that they can recognize instances of WSP traffic and pass the traffic to the WSP parser. Luckily, the Network Monitor team foresaw this need and designed the parser engine so that parsers could communicate with each other.
In any protocol parser, properties are declared and defined in support of protocol state description and management. These properties can be used in capture and display filters as well as any Network Monitor experts. Quite a number of properties are defined for use within the RWS parser to communicate to itself between frames as well as to communicate with the parent (TCP and UDP) protocols. Some of these properties are passed to the WSP parser to help you associate the WSP conversation with the RWS conversation that created them. The primary properties we defined to assist you with troubleshooting Forefront TMG Client-related issues are:
Property Name | Description |
---|---|
Conversation.RwsApp |
the name of the Winsock application, such as “ftp.exe”
Note:
This field is limited to16 characters
|
Conversation.RwsHost |
the name of the computer where Forefront TMG Client is operating, such as “jimsbox” |
Conversation.RwsId |
a binary value derived from the RWS Client Connection Handle value |
Conversation.RwsUser |
the name of the account used to run the Winsock application |
These properties are all defined in the RWS parser and passed to the WSP parser through global property sets. The Network Monitor 3.3 help discusses global properties in detail, so I won’t belabor the point here. The RWS and WSP parsers also include Property.RwsApp, Property.RwsHost, Property.RwsId and Property.RwsUser properties. I’ll describe how to use these when I discuss HTTP via Forefront TMG Client and DNS via Forefront TMG Client in part 2 and 3 of this article series.
All of the RWS parser properties are derived from the RWS messages exchanged between the Forefront TMG Client and Forefront TMG in the process of establishing a traffic path for the client application. Because the majority of this data is internal traffic management or serialized Winsock calls, much of the data in RWS frames may be new to you. Luckily, most problems tend to be simple things such as name resolution failure, attempting to connect an invalid destination IP or port or even normal problems such as an application server listener that isn’t listening. This means you generally won’t have to be concerned with the details of the Winsock calls or how the Forefront TMG Client forwards them to Forefront TMG.
Winsock Applications
In the FWC 101 article, I used an example of an FTP client connection because the FTP protocol is fairly well-understood in the networking community and it illustrates the most commonly-used Winsock calls that generate RWS messages (getaddrinfo(), connect(), bind(), listen(), send(), recv()).
In general, most client-side Winsock applications will fall into a predictable Winsock usage pattern. As a troubleshooter, it’s deviations from these patterns that you will be most interested in identifying. For instance, a Web browser that is not configured to behave as a Web proxy client will perform the following simplified Winsock call sequence:
Resolve the URL hostname to an IP address: getaddrinfo(“hostname”)
Connect to the remote server: connect(ip.add.re.ss:port)
Exchange data on that connection: send(), recv()
These actions translate to the following TGMC and Forefront TMG actions:
Resolve the URL hostname to an IP address
Forefront TMG Client issues gethostbyname(“hostname”) request to Forefront TMG
Forefront TMG resolves “hostname” to an IP address (may call Winsock to do this)
Forefront TMG responds with Host Entry to Forefront TMG Client
Connect to the remote server
Forefront TMG Client issues Connect v12 (ip.add.re.ss:port) request to Forefront TMG. This request also informs Forefront TMG which IP address and port the Forefront TMG Client will use for the application data connection
Forefront TMG connects to ip.add.re.ss:port
Forefront TMG sends Connect v12 response to Forefront TMG Client (includes Forefront TMG side of the application data connection if successful)
Send and receive traffic on that connection
Forefront TMG Client connects to Forefront TMG end of the application data connection
Forefront TMG Client routes outgoing application data through Forefront TMG using the application data connection
Forefront TMG routes incoming data to Forefront TMG Client using the application data connection
Note
You may not see name lookups in the RWS conversation if the client is able to resolve the name using a local DNS service.
Other Winsock applications may exhibit more complex behavior, such as creating listeners for inbound data such as that seen with active FTP or peer-to-peer (P2P) applications or creating multiple outbound connections, such that as seen with may browsers or passive FTP, or perhaps any combination of these.
You can see how this process occurs in a network capture, but only if you use Network Monitor 3.3 and later. Network Monitor 3.3 is currently the only network analysis tool that understands the Forefront TMG Client protocols. You can get the latest Network Monitor 3.3 package and the latest parsers from the links in the References section.
When you are troubleshooting Forefront TMG Client problems, it’s often best to obtain multiple concurrent captures – one at the Forefront TMG Client client computer and one at the Forefront TMG computer. The purpose in doing so is to gather as much data relevant to the two computers as possible in one session. In many cases, a capture taken at Forefront TMG alone could miss a local name query from the client. Likewise, a capture taken at the client alone will not indicate a failed connection attempt by Forefront TMG.
Network Monitor 3.3 Basics
Network Monitor 3.3 improves on Network Monitor 2 as provided with Windows 2000 and Windows 2003 in many significant ways. Parsers are far less likely to cause Network Monitor to fail because of the “sandboxed” design inherent in the parser engine, display and capture filters can be defined much more easily (as well as being imported and exported) and much more. In terms of troubleshooting Forefront TMG Client-managed traffic, the Network Monitor 3.3 Parser Language (NPL) made it much easier to develop and test the RWS parser and in doing so, allowed the Forefront Edge CS team the ability to extend the Network Monitor traffic analysis capability for Forefront TMG Client troubleshooting.
Traffic Capture Methods
I have four general rules for gathering network traffic:
Use command-line tools whenever possible. The extra CPU and memory expense involved with displaying traffic as it is captured can be significant; especially on a host that is already resource-constrained, such as a heavily-tasked Forefront TMG computer. Network Monitor provides nmcap.exe for this very purpose. You and your customers will benefit greatly through your becoming familiar with this tool. In fact, ISABPA and TMGBPA both gather network captures using nmcap.exe.
Use no capture filters whenever possible. Of course there will be exceptions to this rule such as when you already know what protocols you’re chasing, but for the most part, it’s much more effective to gather all of the data extant “on the wire” and filter out what you don’t want to see and change your display filters as your traffic analysis progresses.
Capture the traffic where you can observe the most data at once. Just because the user complains doesn’t mean the interesting data is available at her computer or at Forefront TMG internal NIC. It may be seen only at the external NIC, which means capturing at the client will produce interesting data, but probably nothing that is useful in the current effort.
Try to limit the amount of time spent capturing. Network captures consume disk and memory and the larger they are; the longer it takes to parse them. This may mean that you coordinate the capture and repro efforts with the user, or that you perform the capture and repro yourself.
Display filters
Learning to use Network Monitor display filters is time well spent. The general syntax is clean and lends itself to simplifying display filters that can be applied to multiple scenarios. Most users find themselves operating in the context of the most basic aspects of the protocols they’re chasing; TCP, UDP, ICMP, HTTP, SMTP, etc. While this is a great starting point for drilling down to the problem area, what you’re missing out on when you limit your filtering skills to the base protocol are the finer points of filter mechanisms through the use of protocol properties. For example, a simple display filter trying to identify Web proxy traffic between a client at 10.10.10.10 and a Forefront TMG server at 10.10.10.1 might be defined as:
(ipv4.address==10.10.10.10 and ipv4.address==10.10.10.1) and http
Of course, this display filter excludes any TCP traffic that is not identified as HTTP as well as any DNS or WINS traffic sent by the client that may precede the TCP connection between the client and Forefront TMG. Consequently, if the problem you’re chasing is WPAD failure caused by a name lookup failure for “wpad”, this display filter will result in a Network Monitor packet summary display window that is missing this name query. Understanding the peripheral network processes that are involved with communications between two hosts is critical to solving a problem through network analysis.
Likewise, a simple filter that could help you troubleshoot Web Proxy Automatic discovery (WPAD) might be written as:
HTTP.Request.URI=="/wpad.dat"
If you apply this display filter to a WPAD capture, you’ll see the results indicated in the figure below:
While this display filter might show you all instances where a Web client sent a request for “/wpad.dat”, it won’t show you any response that the application may have received to this request. The HTTP protocol doesn’t specify that the server response must include any part of the original request in the response, so you can’t look for “/wpad.dat” there. You can however look for other data that will be delivered by Forefront TMG when it responds to these requests. If you capture this process and examine the response from Forefront TMG, you will see the following:
- Http: Response, HTTP/1.1, Status: Ok, URL: /wpad.dat
ProtocolVersion: HTTP/1.1
StatusCode: 200, Ok
Reason: OK
Date: Tue, 28 Apr 2009 20:59:37 GMT
Connection: close
- ContentType: application/x-ns-proxy-autoconfig
MediaType: application/x-ns-proxy-autoconfig
Cache-Control: max-age=3000
HeaderEnd: CRLF
+ payload: HttpContentType = application/x-ns-proxy-autoconfig
The Forefront TMG response to a request for the WPAD script is always delivered with a content-type of “application/x-ns-proxy-autoconfig”, making it possible for you to define a display filter that includes the WPAD response as well as the request:
HTTP.Request.URI=="/wpad.dat" or HTTP.Response.HeaderFields.ContentType.MediaType == " application/x-ns-proxy-autoconfig"
Applying this display filter will produce results similar to that shown in the figure below:
Note
Due to the way Network Monitor parses HTTP headers, the header field data must be preceded with a space character in order for the display filter to work when you specify equality (==). You could just as easily use the “contains” method as indicated in the figure below to achieve the same results without the additional space character.
Bear in mind that the filters I’ve just described don’t include the full WPAD protocol, which includes DNS (or WINS) as well as HTTP. The good news is that display filters written for proxy scenarios will soon be part of Network Monitor. Until then, feel free to see what criteria you can identify to help you define your own filters.
Summary
I’ve outlined the RWS command set and discussed the mandatory initial RWS command sequence and useful RWS parser properties. In Part 2, I’ll provide examples of single-connection TCP using HTTP as the application protocol between Forefront TMG Client and Forefront TMG. In part 3, I’ll cover single-connection UDP traffic between Forefront TMG Client and Forefront TMG using DNS as the application protocol.
Tools / Techniques
I’ve compiled a set of useful tools and methodologies to make your RWS troubleshooting go just a tad smoother.
Control Channel Encryption
If you require authentication for even one access rule, ISA Server and Forefront TMG will require RWS encryption after a successful Logon cycle. While this helps to prevent anyone sniffing RWS conversations on the wire, it also prevents you doing the same thing. Needless to say, this makes it somewhat difficult to troubleshoot RWS conversations.
Note
Disabling RWS encryption is a global setting; if you do it at all, you do it for all clients communicating with the affected array.
ISA Server 2004 and 2006
Open the ISA Server management console.
In the left pane, do the following:
In Enterprise Edition, expand Arrays, and then <ArrayName>, in Standard Edition, expand <ArrayName>.
Expand Configuration
Select General.
In the details pane, click Define Firewall Client Settings.
In the Firewall Client Settings window, do the following:
Select Allow non-encrypted Firewall client connections.
Click Apply, and then click OK.
To save your changes, on the Apply Changes bar, click Apply.
FWC
This configuration can create some interesting behavior if you don’t disable it when your troubleshooting is finished. If the FWC has encryption disabled, but ISA Server or Forefront TMG requires it, your FWC applications will not work. To disable encryption for the Firewall Client, you must set some registry values. The commands to accomplish this are listed below.
Note
The quotes indicated in the “reg” commands are mandatory because the registry path includes spaces.
Click Start, All Programs, Accessories, Command Prompt.
Note
For Windows Vista and later, right-click Command Prompt and select Run as Administrator.
To disable Control Channel encryption, in the command window, enter the following command:
32-bit Windows: reg add "HKLM\Software\Microsoft\Firewall Client 2004\Policies" /v WspEncryptControlChannel /t REG_DWORD /v 0 /f 64-bit Windows: reg add "HKLM\Software\Wow6432Node\Microsoft\Firewall Client 2004\Policies" /v WspEncryptControlChannel /t REG_DWORD /v 0 /f
There is no need to restart the firewall or the firewall agent for these changes to take effect.
Be sure to wait until the ISA array has synchronized the policy before starting your tests or confusion will certainly ensue.
When finished testing, re-enable Control Channel encryption with the following command:
32-bit Windows: reg delete "HKLM\Software\Microsoft\Firewall Client 2004\Policies" /v WspEncryptControlChannel /f 64-bit Windows: reg delete "HKLM\Software\Wow6432Node\Microsoft\Firewall Client 2004\Policies" /v WspEncryptControlChannel /f
Forefront TMG and Forefront TMG Client
Because the “Allow unencrypted Firewall client connections” also allowed UDP-based control channel connections and Forefront TMG no longer supports the UDP control channel, this control was provided only in the Forefront TMG scripting COM. To disable control channel encryption, you have to run a script against the array where you want this action to take effect.
VBScript
Option Explicit
Dim oRoot: Set oRoot = CreateObject("FPC.Root")
Dim oArray: Set oArray = Root.GetContainingArray
Dim oClntCfg: Set oClntCfg = oArray.FWClientConfigSettings
oClntCfg.EnableControlChannelEncryption = Not( oClntCfg.EnableControlChannelEncryption )
oArray.Save(true, true)
Save this script as c:\togglerwsencryption.vbs.
Run this script on the Forefront TMG computer from an elevated command window as cscript c:\togglerwsencryption.vbs.
JScript
var oRoot = new ActiveXObject("FPC.Root");
var oArray = oRoot.GetContainingArray();
var oClntCfg = oArray.FWClientConfigSettings;
oClntCfg.EnableControlChannelEncryption = !oClntCfg.EnableControlChannelEncryption;
oArray.Save(true, true);
Save this script as c:\togglerwsencryption.js.
Run this script on the Forefront TMG computer from an elevated command window as cscript c:\togglerwsencryption.js.
Be sure to wait until the Forefront TMG array has synchronized the policy before starting your tests or confusion will certainly ensue.
Unlike ISA Server and the FWC, there is no need to change anything at the client when Forefront TMG is configured this way.
To re-enable control channel encryption, simply re-run the script and wait for the policy to synchronize.
Network Monitor downloads
Network Monitor 3.3 (https://www.microsoft.com/downloads/details.aspx?FamilyID=983b941d-06cb-4658-b7f6-3088333d062f)
Latest Network Monitor 3.3 parser (https://nmparsers.codeplex.com/)
References
Firewall Client Basics: Introduction to the ISA Server Firewall Client and Forefront TMG Client
IANA Port Assignments (https://www.iana.org/assignments/port-numbers)
Microsoft Knowledgebase article 929851 Winsock Ephemeral Port assignments (https://support.microsoft.com/kb/929851/)
Microsoft Knowledgebase article 196271 Ephemeral Port Range Adjustment (https://support.microsoft.com/kb/196271/)
ISABlog on Client / Firewall compatibility (https://blogs.technet.com/isablog/archive/2009/11/03/forefront-tmg-client.aspx)
Network Monitor 3.3 RWS Parser Basics, Part 2: Observing single-connection HTTP traffic
Network Monitor 3.3 RWS Parser Basics, Part 3: Observing single-connection DNS traffic