Load Balancing Microsoft® Speech Server 2004 Enterprise Edition

  • Article
On This Page

Introduction Introduction
Preparing a Speech Server Distributed Deployment Preparing a Speech Server Distributed Deployment
Adding and Load-Balancing TAS Servers Adding and Load-Balancing TAS Servers
Adding and Load-Balancing SES Servers Adding and Load-Balancing SES Servers
Adding and Load-Balancing Web Servers Adding and Load-Balancing Web Servers
Using Third-Party Hardware Load Balancing Using Third-Party Hardware Load Balancing
Conclusion Conclusion

Introduction

Microsoft® Speech Server (MSS) 2004 Enterprise Edition meets call center demands for high performance and high availability with the ability to scale both up and out in its deployment. In a distributed configuration, with multiple Speech Server computers deployed, effective load-balancing of server resources is crucial to handling high call volumes while maintaining excellent performance levels. Speech Server’s support for both hardware and software load-balancing provides enterprises with flexible capacity-handling options.

In this paper we discuss the load-balancing capabilities of Speech Server and offer best-practice guidance and tips for preparing, implementing and verifying load-balanced Speech Server configurations using the Microsoft Windows Network Load Balancing (NLB) service of Windows Server 2003. High-level best practices when using a third-party hardware load balancer are also discussed.

Preparing a Speech Server Distributed Deployment

At a high level, Microsoft Speech Server is composed of two primary components: Telephony Application Services (TAS) and Speech Engine Services (SES). TAS primarily handles the processing of telephony-based speech-enabled Web applications, as well as telephone call routing and receiving; SES handles voice and audio interactions, including speech recognition and text-to-speech functionality. Microsoft Speech Server Standard Edition includes both TAS and SES components on one machine and does not support a distributed configuration or load-balancing of SES resources. Microsoft Speech Server 2004 Enterprise Edition, however, is specifically designed for distributed deployment with the ability to load-balance all the components of a MSS distributed deployment topology.

There are four important benefits that load-balancing Microsoft Speech Server components provides:

  • Increases overall system performance and scalability

  • Protects against single points of failure in the deployed hardware or software

  • Allows seamless twenty-four hours a day, seven days a week operations even when patching or upgrading applications

  • Provides a clear and simple growth path for adding additional servers

In the simplest MSS Enterprise Edition deployment, the TAS component resides on one physical server (the TAS server), SES resides on a separate server (the SES server), and an ASP.NET Web server exists on a third server, as shown in Figure 1 below:

Label, 1.  A basic distributed MSS configuration

Label, 1.  A basic distributed MSS configuration

However, to ensure availability of telephony-based voice-only applications, most organizations will choose to opt for a redundant configuration and run two or more TAS servers and two or more SES servers. In this case, if one TAS server fails or is offline for maintenance, the other TAS server is still available to respond to calls. Additional TAS and SES servers can also be added to accommodate higher call volumes and/or support speech applications of greater complexity.

Since speech processing generally requires more CPU power and memory than the telephony processing performed by TAS, distributed MSS deployments will typically include a greater number of SES servers than TAS servers.1 Routing phone call load to multiple TAS servers takes place in front of the TAS servers, and is thus load-balanced at the telephone switch or Private Branch Exchange (PBX) level. In contrast, the SES servers and Web servers are both load-balanced at the server level by either a software load balancer or a hardware load balancer.

Configuring a Basic Distributed Deployment

Before introducing load-balancing into a MSS deployment, it is prudent to first establish and verify a basic distributed deployment. This basic deployment consists of one TAS server, one SES server, one Web server and no load balancer, as shown in Figure 1 above.2 

A Speech Server deployment is typically configured and managed through the Windows Server 2003 Microsoft Management Console (MMC). An MSS MMC snap-in (Microsoft Speech Server Management Console) is included with your MSS installation, providing easy access to the configuration parameters required for any Speech Server deployment for both SES and TAS components, no matter which machines they’re installed on. Additional servers and services (for example, Web server and the Speech Application Deployment Service (SADS)) can also be added to this MMC template for centralized access to and management of all relevant Speech Server assets.

To establish a basic MSS distributed deployment, use the Microsoft Speech Server Management Console available at Start | All Programs | Microsoft Speech Server 2004 to access and configure the following parameters:

Telephony Application Services Configuration Parameters

  • Inbound application start page. Accessible from the TAS properties dialog box in the MMC, this parameter points to the speech-enabled Web application’s start page and takes the form of https://<WebServerName>/<appfolder>/<startpage>.* *

  • Speech lobby URL. Also accessible from the TAS properties dialog box, this property should be set to the direct hostname of the initial SES server. The default form is https://<SESServerName>/SES/Lobby.asmx.

  • Trusted Sites. When the TAS server is selected within the MMC, the Trusted Sites list is accessible from the Action | All Tasks | Configure Trusted Sites menu option. The Trusted Sites list provides a security layer to prevent unauthorized access to the Microsoft Speech Server components. The list of trusted sites should include the Web URL for the Web server that hosts the speech application, either as an IP address or as a hostname.3 The default form is https://<HostName>.

Speech Engine Services Configuration Parameters

  • Preloaded resource manifest URL. This property points to the resource manifest file, which lists the grammar files and prompt databases to preload on application startup. Preloading resources increases performance for the first client requests. Otherwise, clients must wait while the resources are found and processed.4 The default form is https://<WebServerName>/<appfolder>/manifest.xml.

  • Trusted Sites. As with the TAS server parameter, this points to the Web server URL that hosts the speech application.

  • Access Control List (ACL) permissions. By default, Speech Server is designed to be secure, so access to any SES server, even by a TAS server, is automatically restricted. Only accounts with Read permission to the Access.txt file are granted use of SES. Use the Windows ACL (Access.txt) to specify the account names that should be able to access the SES server. In a domain environment this is handled automatically by the NetworkService account, but this is not supported in a workgroup. For a workgroup, this requires manually adding user accounts to each Microsoft Speech Server computer (see the Microsoft Speech Server Help documentation for more information).

Verifying a Basic Distributed Deployment

To ensure that the basic distributed deployment is working properly, perform the following steps.

  1. In the TAS configuration properties dialog box, confirm that the Speech lobby URL property points to the URL or IP address of the SES server. The default form is* *https://<SESServerName>/SES/Lobby.asmx.5

  2. From within Microsoft Internet Explorer on the TAS server, open the speech lobby page by browsing to the URL shown above.

  3. Confirm that the content resembles the Lobby.asmx page as shown in Figure 2 below. If you’re having trouble accessing this file, check your Internet connection settings (proxy, security, and so on) as well as Web virtual directory settings, ACLs, and so on.

    Label, 2.  Verifying access to the Speech Lobby file

  4. After verifying the lobby.asmx file can be properly accessed via Internet Explorer add the following suggested performance counters to the Windows Server 2003 Performance Monitor:

    Component

    Counter

    Counter Location

    Description and Expected Behavior

    SES

    Current Open Sessions

    Speech Engine Services performance object

    This will show the number of open sessions on the SES server. When a TAS instance has successfully opened a session with a SES instance, this number should increment by one.

    TAS

    Active Incoming SALT Interpreters

    Telephony Application Services performance object

    When a call has been answered and not rejected, this number should increment by one.

    TAS

    Active Outgoing SALT Interpreters

    Telephony Application Services performance object

    For outbound applications, this number should increment by one when a call is placed.

    Web server

    Current Connections

    Web Server performance object

    This will show the number of current connections to the Web server.
    1 SALT is short for "Speech Application Language Tags," a set of lightweight extensions to HTML, XHTML and other markup languages the enable a developer to easily add speech recognition and speech playback (either synthetic or prerecorded) to their telephony or multimodal Web applications.

  5. Place a phone call into the speech application, and monitor the activity to ensure correct behavior, for example, that the number of Active SALT Interpreters increments as an application loads.

  6. Use the Windows Server 2003 Event Viewer to review the TAS server and SES server Event Logs (found under Application events) to verify that no unexpected errors occurred. If errors occurred, double-click the event to view the problem description and attempt to resolve the problem before repeating the verification procedure.

Once a basic distributed deployment has been successfully configured and verified, you can move on to adding more SES and/or TAS servers to your deployment.

Adding and Load-Balancing TAS Servers

As previously mentioned, TAS servers are load-balanced not through IP load-balancing techniques, but instead via the phone switch or PBX.6 

Tip: Because TAS servers are more complex than SES servers, it is better to introduce all TAS servers to the system before load-balancing the SES servers. Diagnosing TAS and other telephony issues is easier when there is only one SES server in the system.

If your deployment contains only one TAS server, proceed to the next section, Adding and Load-Balancing SES Servers. Otherwise, using the Microsoft Speech Server Management Console (by default available from the Start | All Programs | Microsoft Speech Server 2004 fly-out menu), perform the following steps:

  1. Add all remaining TAS servers to the Microsoft Speech Server Management Console view.

    From the Action menu in the Microsoft Speech Server management console you can use the Add/Remove Services to add additional TAS services to your deployment, as shown in Figure 3 below:

    Label, 3.   Adding new servers to the Microsoft Speech Server Management Console template

    Label, 3.   Adding new servers to the Microsoft Speech Server Management Console template

  2. Replicate the initial TAS server settings to the newly added TAS servers.

    Each of the TAS servers in your deployment should use the same configuration settings. Rather than perform this step manually, consider saving the settings from the initial TAS server (as shown below in Figure 4); you can then easily load the settings into the other TAS server instances by choosing Load Settings from the All Tasks context menu.

    Label, 4.  Save and load TAS and SES settings to easily replicate settings across multiple servers

    Label, 4.  Save and load TAS and SES settings to easily replicate settings across multiple servers

  3. Using the MSS MMC snap-in, check the Service Status on each TAS server to verify that all TAS services are in the Running state.

Once the above TAS servers have been configured, and you have configured your phone switch or PBX to load-balance the routing of phone calls coming into the system, you should now verify that Speech Server is still working as expected

Verifying a TAS Server Farm Deployment

To ensure that all TAS servers are working as expected, you should now perform the same verification steps performed earlier on the basic distributed deployment:

  1. In the TAS configuration properties dialog box, confirm that the Speech lobby URL property points to the URL or IP address of the SES server. The default form is https://<SESServerName>/SES/Lobby.asmx.

  2. From within Internet Explorer on the TAS server, open the speech lobby URL.

  3. Confirm that the content of each page resembles the Speech Lobby URL screen shot example as seen in Figure 2 above.

  4. Monitor all suggested performance counters as indicated above when creating a basic distributed deployment. While monitoring performance counters, load the system as necessary to verify the correct operation of each SES, TAS and Web server. This may require some type of load-testing software, or perhaps having a group of people call in to use the system at the same time.

  5. Review the TAS server and SES server Event Logs for errors. If errors exist, use the Description property to determine and rectify the problem(s).

Adding and Load-Balancing SES Servers

Although TAS servers must be load-balanced at the phone switch or PBX, SES servers can be load-balanced at the computer level using NLB to enhance the scalability and availability of speech services. NLB, part of the Cluster Service available in Windows Server 2003, allows administrators to load-balance speech sessions across multiple physical servers for increased scalability, better performance, and fault tolerance. Also, NLB automatically detects if an SES server is unreachable and, if so, removes it from the load-balancing pool.  

Caution: Before stopping or pausing a SES service, be sure to remove the service from the NLB pool first. If a SES service is paused or stopped, NLB will still route requests to it, resulting in application timeouts or errors.

As shown in Figure 5, a load balancer can be placed between TAS servers and SES servers to provide load-balancing services.

Label, 5.   Load-balancing SES servers

Label, 5.   Load-balancing SES servers

Creating Identical SES Servers

This section describes how to begin the creation of an SES server farm by adding a collection of identical SES servers to the deployment configuration.

The following tasks can all be performed using the Microsoft Speech Server Management Console:

  1. Add all SES servers to the system using the Microsoft Speech Server Management Console by right-clicking on the server group and selecting Add/Remove Services.

    Caution: When adding SES servers to the server names list use static IP addresses rather than a virtual server name. Using virtual server names via Dynamic Host Configuration Protocol (DHCP) can result in the unexpected expiration or reassignment of critical IP addresses, thus compromising the integrity of your Speech Server deployment.

  2. Replicate the settings for the initial SES server to all other SES servers.

    Note: The Audio Connection IP and Cache Location properties should not be replicated. These properties must be set manually if they differ from default installation values. Also, some SES properties require a service restart to take effect. If the replication process alters any of these properties, a restart of all affected SES services will be required.7 

  3. Ensure that all SES services are in the running state and have been restarted if required.

Once the SES servers have been created and configured, the next step is to configure Windows NLB to load- balance the speech sessions across these SES servers.

Configuring Each SES Server with Two Network Interface Cards (NICs)

Before discussing how to configure NLB for use with Speech Server, it's important to understand both the hardware recommendations for NLB as well as some of the underpinning processes that take place in a load-balanced MSS deployment.

To increase performance, Microsoft recommends using two NICs for any server that is load-balanced using Windows Network Load Balancing running in unicast mode. One NIC is used for load-balancing communication, and the other NIC handles the actual load to be balanced. This is particularly important when using NLB in unicast mode, as is required when load-balancing SES servers. In Windows Server 2003, NLB masks the Media Access Control (MAC) address of the NICs behind the load balancer, which may result in broadcast storms propagated by switches that depend on the MAC address for optimum routing. Using two NICs is important when dealing with large amounts of audio packets, such as those in Microsoft Speech Server.

TAS issues a request for an SES server by using the IP address or virtual server name defined by the Speech Lobby URL property. In a load-balanced SES server farm, this is the virtual IP address of an SES server farm. The load balancer returns an SES server from its list. This process is referred to as “SES discovery.” In a properly configured SES server farm, there is one dedicated IP address (network interface) on each individual SES server used solely for this discovery. TAS effectively issues GetNewURL requests over these interfaces. Ideally, the SES discovery interfaces should not be resolvable by anything other than the load balancer. Failure to adhere to this resolution rule may result in useless session connections should the SES session network interface fail. Also, all network interface availability should be monitored. This can be done through scripts, WMI event subscription, or by installing monitoring software, such as Microsoft Operations Manager (MOM). For clarity in a production environment, administrators may want to rename these network interfaces “SES Discovery.”

After an SES server has been discovered for a given call, no further communications are routed via the SES discovery network interface. Instead, a session is established directly with the chosen SES server. This session is said to be connected via the SES session network interface. In a properly configured SES farm, there is a dedicated IP address (network interface) on each individual SES server used solely for opening sessions. This IP address is defined on each SES server by the Audio Connection IP property. SES session interfaces should be resolvable using DNS, Hosts file, and so on. They should not be listed in load-balancer pools. For clarity in a production environment, administrators may want to rename these network interfaces to “SES Session” or “SES Audio.”

Also, when more than one NIC is used in an SES server, by default SES arbitrarily chooses one of these NICs on which to route audio traffic. Instead of accepting the default behavior, administrators should specify which IP address to use for audio traffic by manually setting the Audio Connection IP property on each SES server.8 

Configuring Network Load Balancing

This section describes how to configure an SES server farm using NLB. After reading this section and before you enable and configure NLB, be sure to review the NLB documentation available from within the Network Load Balancing Manager application (Help | Help Topics). The NLB documentation includes an NLB configuration checklist with sections that discuss cluster parameters, host parameters and port rules. It is highly recommended that you read through these three NLB documentation topics before configuring NLB: cluster parameters, host parameters, and port rules.

Setting General Cluster Configuration Parameters

From either an SES server or a computer on the network that has adequate permissions to the SES servers, do the following:

  1. Select Start | Administrative Tools | Network Load Balancing Manager to start the Network Load Balancing Manager.

  2. Right-click on the Network Load Balancing Clusters node in the left pane, and select New Cluster. Follow the steps below to configure the cluster settings.

  3. In the Cluster Parameters dialog box, set the following properties as indicated:

    • IP address: Set to the cluster (virtual) IP address you want to use for the SES server   farm

    • Subnet mask: Set to the subnet mask of the Local Area Network under which the SES server farm resides

    • Full Internet name: Set to the fully qualified cluster name of the SES server farm to be load-balanced

    • Cluster operation mode: Unicast

    • Allow remote control: Optional

    Administrators may want to use a virtual server name (often called the cluster name in NLB) in addition to a virtual IP address. Virtual server name to virtual IP address resolution can be accomplished by using DNS or by editing the Hosts file found in the %systemdrive%\WINDOWS\system32\drivers\etc folder.

  4. In the Cluster IP Addresses dialog box, if you’ve created more than one cluster you may add additional cluster IP addresses.

  5. In the Port Rules dialog box, select the All port rule and click on Edit.

  6. In the Add/Edit Port Rule dialog box, set the following properties as indicated:

    • Port range: Restrict the port usage to only used ports (port 80 is default for http requests)

    • Protocols: TCP

    • Filtering mode: Multiple host with Affinity set to None

Adding an SES Server to the Cluster

Follow the steps below to add all SES servers to the cluster.

  1. In the Connect dialog box, enter the computer name of the SES server you want to add in the Host field, and then click on Connect.

  2. The interface names of the Local Area Connection appear in the Interfaces available for configuring a new cluster list view. There will be two entries listed, provided you have the necessary two network interface cards (NICs) in each new server. See “Configuring Each SES Server with Two NICs” above for more information.

  3. Highlight the network interface intended for SES discovery, and then click on Next.

  4. Don’t make any changes in the Host Parameters dialog box.

  5. Click on Finish.

Verifying the Load-balanced Single SES Server Deployment

Before adding more SES servers to the deployment configuration, confirm that load-balancing is working.

To verify that a load balancer has been properly inserted into the configuration and that the single SES server is working in the load-balanced configuration, perform the following steps:

  1. In TAS, change the Speech lobby URL property to point to the virtual IP address of the SES server farm. The default form is https://<VirtualIP>/SES/Lobby.asmx.9

  2. Open the newly defined speech lobby URL on the TAS server from within Internet Explorer.

  3. Confirm that the content resembles the Speech Lobby URL screen shot example as seen in Figure 2.

  4. Monitor all suggested performance counters as indicated above when creating a basic distributed deployment. While monitoring performance counters, load the system as necessary to verify each SES, TAS and Web server.

  5. Review the TAS server and SES server Event Logs for errors.

    Note: At the beginning of a telephone call, the TAS server attempts to contact the SES server farm, which may consist of one or more SES servers. At this stage, the TAS server cannot distinguish between a direct SES server and a virtual SES server. Therefore, errors that are reported in SES server discovery are general and do not contain specific server name information. Instead, the Event Logs may report that an SES server could not be found. This can mean either that the load balancer (that is, the virtual SES server) or one or more of the SES servers could not be located. It does not necessarily mean that all the SES servers are unreachable.

Adding Additional SES Servers to the Cluster

  1. Add all remaining SES servers to the cluster, right-click on the SES server farm (cluster name) in the NLB Manager and select Add Host To Cluster, then repeat the steps in this section.

  2. Wait until each host has converged before adding another host. The log panel in the bottom pane of the NLB Manager should display “End configuration change” and show no errors. The reported status may lag behind the actual status. Press F5 in the NLM Manager to refresh cluster status.

  3. To check on the status of a particular host, log on to the host and type “NLB QUERY” at a command prompt.

  4. Verify that the SES server has been added to the cluster by expanding the Network Load Balancing Clusters listing in the left pane of the NLB Manager.

Verifying a Load-balanced SES Server Farm Deployment

To ensure that the distributed deployment is working as expected, perform the following series of steps:

  1. In TAS, confirm that the Speech Lobby URL property points to the virtual IP address of the SES server farm. The default form is https://<VirtualIP>/SES/Lobby.asmx.

  2. Open the Speech Lobby URL on the TAS server from within Internet Explorer.

  3. Confirm that the content resembles the Speech Lobby URL screen shot example as seen in Figure 2 above.

  4. Monitor all suggested performance counters as indicated above when creating a basic distributed deployment. While monitoring performance counters, load the system as necessary to verify each SES server.

  5. Review the TAS server and SES server Event Logs for errors.

  6. Remember to remove SES servers from the load-balancer pool before pausing or stopping the SES services on a given server. Failing to do so may result in call-session errors.

Adding and Load-Balancing Web Servers

As shown in Figure 6 below, the strategy for configuring a load-balanced Web server farm hosting is similar to the approach used to load-balance an SES server farm.

Label, 6.   Load-balancing the Web servers

Label, 6.   Load-balancing the Web servers

Please note the following exceptions and comments:

  • Although it is not necessary in all cases, it is good practice for every SES server to have the virtual server site and each Web server site name listed in its MSS Trusted Sites list.

  • Although it is not necessary in all cases, it is good practice for every TAS server to have the virtual server site and each Web server site in its MSS Trusted Sites list.

  • In contrast to MSS, Web servers are not limited to round-robin methodology. Depending on your applications, other load-balancing algorithms may be used in lieu of round-robin.  

  • Placing a firewall between MSS and the Web servers is supported.

  • If multiple applications are being used, and therefore the Speech Application Deployment Service (SADS) is employed, it is not possible to automatically replicate the SADS properties across all Web servers. Instead, the applist.xml file found in the %systemdrive%/InetPub/wwwroot/MS_Speech_Deployment folder must be manually copied from a master Web server to all other Web servers.

  • If application affinity does not need to be maintained from MSS to Web servers, the resource manifest locations found in the SADS UI can point to the load-balanced cluster name. This effectively uses a recursion method to obtain resources for preloading, resulting in a more even load across a Web server farm. However, this may not be appropriate for every application or deployment scenario. Notice the difference between these two forms in the paths below:

    URL Form

    Behavior

    https://<WebServerCluster>/<VirtualDirectory>
    /ApplicationResourceFile.xml

    Recursively contacts the Web server cluster

    https://<LocalWebServer>/<VirtualDirectory>/
    ApplicationResourceFile.xml

    Maintains affinity between Microsoft Speech Server and the Web servers

Verifying a Fully Load-Balanced MSS Distributed Deployment

To ensure that the load-balanced Web server farm is working as expected, perform the following verification steps, similar to those detailed in previous sections.

Note: Although the first three steps apply only indirectly to the Web servers, it is always a good idea to verify the entire system.

  1. In TAS, confirm that the Speech Lobby URL property points to the virtual IP address of the SES server farm. The default form is https://<VirtualIP>/SES/Lobby.asmx.

  2. Open the Speech Lobby URL on the TAS server from within Internet Explorer.

  3. Confirm that the content resembles the Speech Lobby URL screen shot example as seen above in Figure 2.

  4. Monitor all suggested performance counters as indicated above when creating a basic distributed deployment.

    For Web servers, monitor “Current Connections” found in the Web Server performance object. Although the actual number of connections may not be informative, the distribution of connections should be nearly even across a properly load-balanced Web server farm.

    Note: If the load is configured correctly but the number of Current Connections is still not balanced as expected, try restarting Internet Information Services (IIS) by typing iisreset in a command window. This will clear out the IIS-cached resources.

    While monitoring performance counters, load the system as necessary to verify each Web server.

  5. Review the TAS server and SES server Event Logs for errors.

Using Third-Party Hardware Load Balancing

Companies can use third-party hardware load balancers in place of NLB to load-balance MSS distributed deployments. The following are recommended best practices when using third-party hardware load balancers:

One-Armed Configuration for Speech Server

Because the MSS Enterprise Edition model requires that TAS servers and SES servers be able to connect directly, a one-armed, load-balanced configuration is highly recommended for TAS-to-SES traffic when using a hardware load balancer. This recommendation does not extend to communication with load-balanced Web server farms.  

One-armed configuration allows the initial load-balancing request to travel between the load balancer and target. Subsequent transactions are direct from server to server, bypassing the load balancer. TAS can only send and receive audio through a direct SES server—not through a virtual SES server. Hence, a one-armed configuration lends itself well to an MSS deployment.

Block Unused Port Settings

All unused ports through load balancers should be blocked. By default, port 80 is used for http requests between TAS, SES and SADS. Optionally, port 443 can be used for https requests.

Use Unweighted Round-Robin

Unweighted round-robin is the only load-balancing scheme recommended for MSS services. Any other algorithms may lead to undesired results when service states change, when services are too busy or when connectivity issues arise. Weighted round-robin may be used as long as the risks are understood and accepted.

Load-balancing is performed only when a call is first answered. For each call, TAS attempts to connect to a single SES server. Because TAS can only open a successful session with a direct SES server, it must somehow discover one. TAS does this by issuing a GetNewURL request to the speech lobby URL, which in turn selects an SES server to pass the request. The selected SES server returns its hostname back down the path to the requesting TAS.

If a URL is returned, TAS attempts to establish a connection directly, bypassing the virtual SES server. If an SES service declines the request, or if there is no response, TAS issues another GetNewURL request to the virtual SES server. TAS repeats this process four times (for a total of five requests) before rejecting the call.

If weighted round-robin is used for load-balancing, a risk lies in the statistical distribution of the TAS GetNewURL requests to the SES services. In a malfunctioning server situation, a heavily weighted server may produce more rejected calls than an evenly weighted server.

Note: Round-robin load-balancing does not always choose the next server in the pool. Not all load balancers work identically, and an observer may notice deviations from a true round-robin algorithm.

Conclusion

Through its support for a distributed deployment, Microsoft Speech Server 2004 Enterprise Edition provides unlimited scaling capabilities capable of supporting thousands of customer phone calls. Distributing call volumes and speech sessions to multiple servers that host Speech Server requires load-balancing both TAS and SES, and in some cases even multiple Web servers. Using common load-balancing techniques for Speech Server, including load-balancing TAS with the PBX or phone switch and load-balancing SES and the Web servers using Windows Network Load Balancing, an administrator can create and manage a Speech Server deployment that can handle high call volumes with high availability and redundant fail-over capabilities.

1 Multimodal applications are not processed through the telephone and do not require TAS, but they do require at least one SES server, and the SES load-balancing techniques discussed herein apply to both telephony and multimodal speech applications.
2 For the purposes of this paper, this TAS server and SES server are called the initial TAS server and initial SES server, respectively.
3 To ensure that applications will run correctly, when specifying URLs anywhere within the MSS deployment configuration, be consistent and always use either IP addresses or hostnames. For example, if an MSS property is set with a hostname and the Trusted sites list contains only that host’s IP address, Speech Server may not make the connection.
4 For a multiple application environment, this should always be SESManifest.aspx, located on the Web server.
5 If Microsoft Speech Server servers are members of a domain that authenticates in Windows 2000 Mixed mode, a virtual server name must be used instead of a virtual IP address. In Windows Server 2003, you may use either a virtual IP address or virtual server name.
6 Please consult your PBX documentation to determine how to load-balance incoming phone numbers to ports that may span multiple trunk cards within the PBX. These ports or trunk cards will connect to the telephony cards installed in the TAS servers. The documentation should include how to configure a PBX for automatic routing to different TAS computers in the case of computer failure (also known as setting up a hunt group), and best practices for maintaining service availability. Sometimes this facility will be supplied by your telecom provider using Centrex, a network-based switching and telecom service solution offered by many telecom providers. Please consult your telecom provider for more information about this service.
7 See the Microsoft Speech Server Help documentation for SES properties that require a restart.
8 The Audio Connection IP is actually used for more than transporting audio, however, most of the traffic is assumed to be of audio format.
9 As previously mentioned if the MSS servers are a member of a domain authenticating in Windows 2000 Mixed mode, a virtual server name must be used instead of a virtual IP address.