Chapter 13 - Third-Party Load Balancer Support
This chapter provides the information that you need for integrating several third-party load balancers with Microsoft Application Center 2000 (Application Center). The product team was able to work with and test a cross-section of representative load-balancing devices, which are documented in this chapter.
Note The devices that are documented do not reflect any preferences on the part of Microsoft, and nothing should be inferred from the selection of devices that were tested.
Application Center supports integration with the third-party load-balancing devices that we tested by providing a command-line tool and a service that facilitates communication between its own feature set (and user interface) and each of these devices. This Microsoft Application Center 2000 Resource Kit tool was developed as an interim solution in response to customer needs.
Integration with the following devices is currently supported:
Alteon WebSystems 180E
Cisco Systems LocalDirector
F5 Networks BIG-IP
Intel NetStructure 7175 and 7185
On This Page
Conceptual Overview and Functional Requirements
Application Center and Load Balancers—Concepts
Device Monitors
The ACEXT.EXE Command-Line Tool
Device Set Up
Conceptual Overview and Functional Requirements
.gif "Bb734906.spacer(en-us,TechNet.10).gif")
In order to achieve a modest level of integration between Application Center and a load-balancing device, two requirements have to be possible. First, you have to be able to monitor the device from the Application Center user interface; and second, you have to be able to affect a change on the device via the Application Center interface.
The essential functional requirements for load balancer integration are summarized in Table 13.1.
Table 13.1 Functional Requirements for Third-Party Load Balancer Integration
Requirement |
Description |
|---|---|
Obtaining cluster membership status |
Obtaining the status of a cluster member is essential for integration. The user interface needs to know whether a member is currently online or offline in the context of load balancing. This information is used to fire the correct Microsoft Windows Management Instrumentation (WMI) events, trigger the right actions, and inform the user of the current state of the member. |
Setting a cluster member online |
This allows the user to manually bring a member into the load-balancing loop. Supporting this function means that Application Center can also bring a member online automatically when appropriate (for example, when a new member joins the cluster), provided that the new member was configured with load-balancing device support. |
Setting a cluster member offline |
This allows the user to manually remove a member from the load-balancing loop. Supporting this function means that Application Center can also take a member offline automatically in certain circumstances (for example, when a monitor fires an event indicating that there is a hardware/software problem with a member). |
Figure 13.1 provides a high-level architectural and process view that illustrates how functional requirements for integration are satisfied by enabling device monitoring and management.
.gif "Bb734906.f13uj01(en-us,TechNet.10).gif")
Figure 13.1 Third-party load balancer integration architecture
Let's examine the device monitoring aspect of the architecture shown in Figure 13.1.
Device Monitoring
The leftmost part of Figure 13.1 shows the process flow among the major elements in device monitoring: the load balancer, the ACLBDevMon service, and Application Center, notably AC.EXE and Cluster Services.
The ACLBDevMon service polls the device every 60 seconds to determine if a member is online (enabled) or offline (disabled). If the server state changes on the device (for example, from Enabled to Disabled), ACLBDevMon executes AC.EXE with the appropriate Set Online/Set Offline command, which in turn updates Cluster Services with the new state information. From an Application Center perspective, the member is now online or offline.
Device Management
The rightmost part of Figure 13.1 shows the process flow among the major elements in device management: the Application Center user interface, Microsoft Health Monitor 2.1, the device, and the Application Center cluster.
Through the user interface we can initiate a Set Offline/Set Online command against a member, which generates an event that is passed to Health Monitor. In turn, Health Monitor executes ACEXT.EXE (with the necessary parameters), which sends the appropriate command to the device.
Let's step through the process in more detail, using a hypothetical Set Offline action on a cluster member.
In the Application Center snap-in, right-click the membername node, and then click Set Offline.
The user interface communicates with Cluster Services, which generates the following WMI event:
MicrosoftAC_Cluster_LoadBalancing_ServerOfflineRequest_Event
This event is picked up by the LB Device Offline Request Event data collector and the threshold #of Instances Collected > 0 is changed to 1, which triggers a change in server state to Warning.
The Warning state in turn initiates the action LB Device SetOffline, which executes the following command-line instruction:
Acext.exe /Command SetOffline
The ACEXT.EXE command-line instruction is executed on the device, and the target member is disabled—that is, set offline.
Application Center and Load Balancers—Concepts
Application Center uses a slightly different representation and terminology for clusters than that used by the different load-balancing device manufacturers. The principle difference between the way that Application Center views a cluster and the way a load-balancing device views a cluster is the absence of a server-grouping concept.
Note The Application Center view of a cluster is due to the fact that Windows Network Load Balancing does support load balancing of multiple virtual IP addresses. However, it does not implement port differentiation, which is to say, having one virtual IP address with two different ports, each one representing a different cluster.
The other notable conceptual difference relates to services and members. Table 13.2 provides a mapping between the Application Center concept of a cluster, service (for example, HTTP and FTP), and member and the supported devices.
Table 13.2 ApplicationCenter and Device Conceptual Mapping
Application Center |
Alteon WebSystems 180E |
Cisco Systems LocalDirector |
F5 Networks BIG-IP |
Intel NetStructure 7175 and 7185 |
|---|---|---|---|---|
Cluster |
Virtual Server |
Virtual Server |
Virtual Server |
Policy Group |
Service(1) |
Real Server Group |
Pool |
Service |
|
Member |
Real Server |
Real Server |
Member |
Server |
1 This release of Application Center does not support the concept of a service.
The following sections provide an overview of each device and show how each maps to the Application Center concept of a cluster and its members.
Alteon WebSystems 180E
Figure 13.2 shows how Alteon WebSystems 180E device settings map to the main elements of an Application Center cluster: the cluster, the virtual or cluster IP address, and cluster members.
.gif "Bb734906.f13uj02(en-us,TechNet.10).gif")
Figure 13.2 Architectural mapping between ApplicationCenter and the Alteon WebSystems 180E load balancer
Table 13.3 summarizes the Alteon WebSystems 180E server management entities and their available configuration options.
Table 13.3 Server Management Entities for the Alteon WebSystems 180E Device
Entity |
Configuration options |
|---|---|
Virtual Server |
Server name |
IP address |
|
Port number |
|
Pool(s) |
|
Real Server Group |
Name |
Load-balancing mode |
|
Members |
|
Real Server |
IP address |
Port number |
|
Load-balancing weight |
Cisco Systems LocalDirector
Figure 13.3 shows how Cisco Systems LocalDirector device settings map to the main elements of an Application Center cluster: the cluster, the virtual or cluster IP address, and cluster members.
.gif "Bb734906.f13uj03(en-us,TechNet.10).gif")
Figure 13.3 Architectural mapping between ApplicationCenter and the Cisco Systems LocalDirector load balancer
Table 13.4 summarizes the Cisco Systems LocalDirector server management entities and their available configuration options.
Table 13.4 Server Management Entities for the Cisco Systems LocalDirector Device
Entity |
Configuration options |
|---|---|
Virtual Server |
Name |
IP address |
|
Port number |
|
Pool(s) |
|
Real Server |
IP address |
Port number |
F5 Networks BIG-IP
Figure 13.4 shows how F5 Networks BIG-IP device settings map to the main elements of an Application Center cluster: the cluster, the virtual or cluster IP address, and cluster members.
.gif "Bb734906.f13uj04(en-us,TechNet.10).gif")
Figure 13.4 Architectural mapping between ApplicationCenter and the F5 Networks BIG-IP load balancer
Table 13.5 on the following page summarizes the F5 Networks BIG-IP server management entities and their available configuration options.
Table 13.5 Server Management Entities for the F5 Networks BIG-IP Device
Entity |
Configuration options |
|---|---|
Virtual Server |
Name |
IP address |
|
Port number |
|
Pool(s) |
|
Pool |
Name |
Load-balancing mode |
|
Members |
|
Member |
IP address |
Port number |
|
Load-balancing weight |
|
Priority |
Intel NetStructure 7175 and 7185
Figure 13.5 shows how Intel NetStructure 7175 and 7185 device settings map to the main elements of an Application Center cluster: the cluster, the virtual or cluster IP address, and cluster members.
.gif "Bb734906.f13uj05(en-us,TechNet.10).gif")
Figure 13.5 Architectural mapping between ApplicationCenter and the Intel NetStructure 7175 and 7185 load balancer
Table 13.6 summarizes the Intel NetStructure 7175 and 7185 server management entities and their available configuration options.
Table 13.6 Server Management Entities for the Intel NetStructure 7175 and 7185 Device
Entity |
Configuration options |
|---|---|
PolicyGroup |
Name |
Service(s) |
|
Service |
Virtual IP address |
Port number |
|
Balancing mode |
|
Server(s) |
|
Server |
IP address |
Port number |
Device Monitors
A MOF file is required to install the necessary data collectors and actions in the Application Center namespace. This file is Lbdevicehm.mof.
Lbdevicehm.mof
Lbdevicehm.mof is used to create the monitors, data collectors, thresholds, and actions that are required to enable third-party load-balancer device support. These monitors are local monitors and are installed in the Non-Synchronized Monitors group when Lbdevicehm is MOF compiled. Table 13.7 highlights the Health Monitor configuration information that is installed on a member.
Note It is not necessary to modify this file directly if you edit the setup.bat file and insert the necessary configuration information. This batch file also handles the mofcomp of lbdevicehm.mof so it isn't necessary to do this as a separate step.
Table 13.7 Load-Balancer Device Monitors
Entity |
Description |
|---|---|
LB Device Online Request Event |
An event query collector with these properties: EventId, ServerName, Type, CollectionErrorCode. |
LB Device Offline Request Event |
An event query collector with these properties: EventId, ServerName, Type, CollectionErrorCode. |
LB Device SetOnline |
A MicrosoftHM_ActionConfiguration instance to set a member online. |
LB Device SetOnline |
A MicrosoftHM_ActionConfiguration instance to set a member online. |
The ACEXT.EXE Command-Line Tool
With all of the commands, which parameters are necessary and the exact format of the parameters depends on the device being communicated with. The Alteon WebSystems 180E, for example, refers to servers by number (from 1 through 255) and server groups in the same way. For F5 Networks BIG-IP, names are not used for servers and only IP addresses are used. On the Cisco Systems LocalDirector, computer names can be bound to IP addresses (thus, using computer names depends on these being set up on the device). The differences are illustrated in Table 13.8.
Table 13.8 Device Differences for the ACEXT Parameter
Device |
ClusterName |
Service |
ServerName |
ServerPort |
|---|---|---|---|---|
Alteon WebSystems 180E |
Number |
Not used |
Number |
Not used |
F5 Networks BIG-IP |
Pool name |
Not used |
IP address |
Not used |
Intel NetStructure 7175 and 7185 |
Policy group name |
Service name |
Computer name |
Server port |
Cisco Systems LocalDirector |
Virtual server group |
Not used |
Computer name (if configured on device) |
Not used |
Table 13.9 provides all of the ACEXT.EXE parameter information, including valid values and descriptions.
Table 13.9 ACEXT.EXE Command-Line Parameter Information
Switch |
Valid values |
Description |
|---|---|---|
/COMMAND |
GetStatus |
Returns the status of a particular member in a service within a cluster. Possible return values are ENABLED, DISABLED, and UNKNOWN. |
SetOffline |
Sets the member to the Offline state so that no load is directed to it. |
|
SetOnline |
Sets the member to the Online state so that it can start receiving load. |
|
/DEVICE |
BIGIP |
F5 Networks BIG-IP: the supported operating system is version 3.3. |
ALTEON180E |
||
LOCALDIRECTOR |
||
INTEL |
||
/USER |
Valid user name for the device |
|
/PASSWORD |
Valid password for the device |
|
/DEVICENAME |
Host name or IP address for the device |
|
/DEVICEPORT |
Telnet port number; defaults to 23 |
|
/SAVE |
Device, Cluster, Service, and Server details |
Used in combination with other parameters to save a member's entire configuration to the registry. |
/CLUSTERNAME |
Cluster name |
|
/CLUSTERIP |
Cluster/virtual IP address |
|
/CLUSTERPORT |
Cluster port number |
|
/SERVICENAME |
Service name (Pool, Real Server Group, Service, and so on) |
|
/SERVERNAME |
Server name (computer name, real server name) |
|
/SERVERIP |
Server IP address |
|
/SERVERPORT |
Port number that is load balanced |
Using ACEXT.EXE
The following sections describe how you can use ACEXT.EXE to save device configuration details, obtain server status information from a load-balancing device, and set a member online or offline via a device.
Saving Server Configuration Information
You can store all of the server configuration information in the registry by using ACEXT.EXE. Every configuration parameter can be saved by using ACEXT.EXE /SAVE. This removes sensitive information from easily accessible MOF files, and, in the case of DevicePassword, provides greater security because the password is encrypted. (However, remember that Telnet communication is all in plain text.)
From the Windows 2000 command prompt, type
ACEXT /DEVICE ALTEON180E /DEVICENAME MYDEVICE /USER CLUSTERADMIN /PASSWORD ADMINPASSWORD /DEVICEPORT 23 /CLUSTERNAME MYCLUSTER /CLUSTERIP 207.46.130.14 /CLUSTERPORT 80 /SERVERNAME 192.168.100.100 /SERVERIP 192.168.100.100 /SERVERPORT 80 /SERVICENAME 80 /SAVE
Note The preceding example specifies the Alteon WebSystems 180E device, but you can substitute this value with any valid device name, as specified in Table 13.9.
After this command is executed, this information doesn't have to be specified again. You can verify that this information is safely stored in the registry by using ACEXT.EXE to obtain server status information. Execute the following command to obtain configuration status information:
ACEXT.EXE /COMMAND GETSTATUS
The GETSTATUS parameter can also be used with other parameters to obtain server status information for all the cluster members.
Obtain Server Status
You can obtain the current status of any server in the cluster by running ACEXT.EXE with the following parameters and values:
ACEXT.EXE /COMMAND GETSTATUS /CLUSTERNAME MYPOOL /CLUSTERIP 207.46.130.14 /SERVERNAME 192.168.100.100 /SERVERIP 192.168.100.100
This command generates the following output for the specified server:
Microsoft Application Center Load Balancing Device Command Line Utility Microsoft (c) 2000 Results of command: GetStatus Status of server: COMPUTER10AS Status: DISABLED
Set a Member Online/Offline
You can set a member online or offline by using ACEXT.EXE with the following parameters and values:
ACEXT.EXE /SETONLINE /CLUSTERNAME MYPOOL /CLUSTERIP 207.46.130.14 /SERVERNAME 192.168.100.100 /SERVERIP 192.168.100.100
To set the member offline, use the preceding command syntax and replace /SETONLINE with /SETOFFLINE.
Load-Balancing Devices and Draining
Application Center has the concept of draining a member of active connections when it sets the member offline. For a specified period—the drain time—existing connections are maintained, but the member that is going offline does not accept any new connections.
After the specified drain time has finished, the member is taken completely offline for load balancing and any remaining connections are dropped.
Most external load-balancing devices do not have this concept of a drain time; they simply persist the existing connections on a server and redirect new connections to other servers. This is something that you should be aware of when using ACEXT.EXE in combination with AC.EXE to write scripts that shut down services such as Internet Information Services 5.0 (IIS). In these cases, you can simulate a drain time by putting a timed sleep interval in the script. You can insert this sleep interval between commands that disable the member on the device and commands that act on the member in the cluster context.
Device Set Up
This section provides general setup information as well as device-specific configuration steps.
Install the Load-Balancer Programs and Files
To install the tools on a cluster member, you have two options. You can install the Application Center 2000 Resource Kit (the default installation is to the C partition). Additionally, you can copy the device files that you need from the Application Center CD (in the directory Third Party Load-Balancing Tools) to a specific installation directory or to the Application Center installation directory.
You can edit and run the Setup.bat file that is included for each device. This batch file applies the configurations described in the following sections. These actions should be taken after the cluster is created initially on the controller or after a server is added to the cluster. This is necessary to ensure that the appropriate settings are applied to Application Center.
Note To use the Set Online and Set Offline options in the Application Center user interface, the cluster controller needs to be restarted. However, device integration with Application Center will work, provided that you are not using the user interface to set a member online or offline.
The following Setup batch file shows how to configure device support, using the Alteon WebSystems 180E as an example.
Setup Batch File: Alteon WebSystems 180E Device
You need to implement the following edits in this file:
Modify the Application Center installation directory path to reflect the installation path for Application Center on the member.
Change the details for device, cluster, service, and server to reflect your settings.
When finished, the instructions in Setup.bat will be similar to the following example.
acext /Save /Device ALTEON180E /DeviceName myalteon /DevicePort 23 /User username /Password password /ClusterName 1 /ClusterIP 192.168.123.8 /ClusterPort 80 /ServerName 4 /ServerIP 192.168.149.4 /ServerPort 80 /ServiceName http aclbdevmon.exe -service mofcomp lbdevicehm.mof updatepath.vbs "%HOMEDRIVE%\Program Files\Microsoft Application Center" copy acext.exe "%HOMEDRIVE%\Program Files\Microsoft Application Center\." rem ****************************** rem put this in the hosts file: x.x.x.x myalteon rem
Configure the Alteon WebSystems 180E, Intel NetStructure 7175 and 7185, and Cisco Systems LocalDirector Devices
Communication between these devices and servers is via the Telnet protocol. You have to configure the devices as described in the following sections.
On the Device
Enable the Telnet access to allow Telnet sessions from the servers that will be running Application Center. (Refer to the specific load-balancing device documentation).
Because the number of concurrent Telnet sessions is limited on some of these devices, it is possible to lock up Telnet sessions with administrative tasks. There is retry logic in ACEXT.EXE and in the WMI provider to handle this situation; if the number of Telnet sessions is configurable on the device, they should be set to the maximum permissible number. The Telnet configuration on the device should also:
Have the prompt for the command line interface set to the default value.
Disable page scrolling, which is enabled by default.
Configure the F5 Networks BIG-IP Device
Verify that you are using F5 Networks BIG-IP operating system version 3.3 and that the AC2000.CGI has the correct permissions—it does not by default. Set the correct permissions with the following command:
CHMOD 4755 AC2000.CGI
Implement the following configurations on each load-balanced server.
Install the F5 Networks BIG-IP Server Certificate
This device communicates with load-balanced servers with HTTPS by using Secure Sockets Layer (SSL). You have to install the F5 Networks BIG-IP server certificate on each server. Follow these steps to install the certificate on a server:
Copy the F5 Networks BIG-IP server certificate to the server.
Refer to the F5 Networks BIG-IP documentation for instructions on how to generate this certificate.
Install the server certificate by using CERTMGR.EXE as follows:
CERTMGR –ADD –C certificate file -S -R localmachine root
Modify the Hosts File or DNS
Modify the Hosts file or DNS to specify a name that resolves to the device configuration.
Note The host name that you provide must match the name on the server certificate that you created.
Configure All of the Devices
The following device configuration information must be provided for the device that you're using.
Modify lbdevicehm.mof
You have to be sure to provide the appropriate information for the LB Device Server Monitor and CommandLineEventConsumer instances.
Note You can implement the edits that are described directly in Lbdevicehm.mof, but the recommended approach is to edit Setup.bat and enter the required configuration information there.
CommandLineEventConsumer Instances
The CommandLineEventConsumer instances that are associated with LB Device SetOnline Request Event and LB Device SetOffline Request Event need to be modified. Specifically, ExecutablePath needs to be modified. The following code from Lbdevicehm.mof shows the modified instance information in boldface.
instance of CommandLineEventConsumer
{
CommandLineTemplate = "acext.exe /Command SetOnline";
CreatorSID = {1, 5, 0, 0, 0, 0, 0, 5, 21, 0, 0, 0, 160, 101, 207, 126, 120, 75, 155,
95, 231, 124, 135, 112, 254, 92, 1, 0};
ExecutablePath = "D:\\Program Files\\Microsoft Application Center\\acext.exe";
KillTimeout = 120;
Name = "{C92D5871-4E41-47FA-AEF6-24E7D75D3CA5}";
RunInteractively = TRUE;
ShowWindowCommand = 0;
WorkingDirectory = "";
};
instance of CommandLineEventConsumer
{
CommandLineTemplate ="acext.exe /Command SetOffline";
CreatorSID = {1, 5, 0, 0, 0, 0, 0, 5, 21, 0, 0, 0, 160, 101, 207, 126, 120, 75, 155,
95, 231, 124, 135, 112, 254, 92, 1, 0};
ExecutablePath = "D:\\Program Files\\Microsoft Application Center\\acext.exe";
KillTimeout = 120;
Name = "{EA89738D-4AC8-4281-827A-A4AE1735D491}";
RunInteractively = TRUE;
ShowWindowCommand = 0;
WorkingDirectory = "";
};
Note /ServerIP 192.168.100.100 in the preceding .mof file instructions is for illustration only.
Register the Data Collectors and Actions
The final step in setting up the load balancer is registering the custom data collectors and actions that Health Monitor uses for the device. From the Windows 2000 command prompt, or by using Setup.bat, run the following statement:
MOFCOMP LBDEVICEHM.MOF
Now that you've completed this set up, device support is enabled for the Application Center cluster environment. Figure 13.6 shows the LB Device Offline Request Event Properties dialog box.
.gif "Bb734906.f13uj06(en-us,TechNet.10).gif")
Figure 13.6 The LB Device Offline Request Event Properties dialog box
The other monitors that are added after you configure a load-balancing device are LB Device Offline Request Event and LB Device Online Request Event.
Register the ACLBDevMon Service
Use the following command to register the ACLBDevMon service if you're not using Setup.bat:
ACLBDEVMON.EXE -SERVICE
Troubleshooting
Other than incorrect configuration information, such as an IP address, there are common problems that you might encounter. The following set-up issues might arise.
Alteon WebSystems 180E, Cisco Systems LocalDirector, and Intel NetStructure 7175 and 7185 devices:
- The load-balanced server does not have its Telnet service started.
FT Networks BIG-IP device:
The cluster name references the pool name on the device, not the virtual server.
The server certificate isn't installed.
The host name is not in the Hosts file or DNS.
The host name doesn't match the server certificate name.
The operating system version is incorrect.
Ac2000.cgi does not have the correct permissions.