Click to Rate and Give Feedback
TechNet
TechNet Library
Deploying Microsoft Office Communications Server 2007

How Microsoft IT Deployed Office Communications Server 2007 and Office Communicator 2007

Technical White Paper

Published: March 11, 2008

Download

Download Technical White Paper, 1.15 MB, Microsoft Word file

PowerPoint PowerPoint Presentation, 2.70 MB, Microsoft PowerPoint file

Situation

Solution

Benefits

Products & Technologies

As part of its commitment to customers, Microsoft IT validates the latest software builds and design options for Office Communications Server 2007 and Office Communicator 2007 in a production environment. With the experiences gained from deploying and operating the systems, Microsoft IT can offer practical examples and best practices to customers.

Microsoft IT deployed Office Communications Server 2007 and Office Communicator 2007 to employees at Microsoft while the products were still in development. These early users provided valuable feedback to the product groups and helped to ensure that the products would meet availability, scalability, and security requirements before RTM.

  • Enterprise Voice capabilities
  • Enhanced Web collaboration experience and reduced associated costs
  • Enhanced group IM and real-time presence
  • External Microsoft users can access text, audio, and video communications, without having to access the corporate network through VPN
  • Simplified administration of Office Communications Server and reduced total cost of ownership
  • Microsoft Office Communications Server 2007
  • Microsoft Office Communicator 2007
  • Microsoft Office Live Meeting
  • Microsoft Office Communicator Web Access 2007
  • Conferencing Add-in for Microsoft Office Outlook
  • Microsoft RoundTable
  • Microsoft SQL Server
  • Windows Server 2003
  • Active Directory
On This Page

Cc297283.arrow_px_down(en-us,TechNet.10).gif Executive Summary
Cc297283.arrow_px_down(en-us,TechNet.10).gif Communications Infrastructure
Cc297283.arrow_px_down(en-us,TechNet.10).gif Opportunities with Office Communications Server 2007
Cc297283.arrow_px_down(en-us,TechNet.10).gif Design Challenges and Considerations
Cc297283.arrow_px_down(en-us,TechNet.10).gif Office Communications Server 2007 Implementation
Cc297283.arrow_px_down(en-us,TechNet.10).gif Operations and Support
Cc297283.arrow_px_down(en-us,TechNet.10).gif Lessons Learned
Cc297283.arrow_px_down(en-us,TechNet.10).gif Best Practices
Cc297283.arrow_px_down(en-us,TechNet.10).gif Conclusion
Cc297283.arrow_px_down(en-us,TechNet.10).gif For More Information

Executive Summary

Companies of all sizes, and especially Fortune 500 companies with global presence such as Microsoft, increasingly rely on unified communications systems as a way to provide workers with flexible workplaces and the latest communication technologies, and to reduce the costs of operating facilities. The Microsoft Information Technology (Microsoft IT) group is responsible for ensuring that the 130,000 Microsoft users worldwide—partners, vendors, employees, and associates—have access to the best available communication tools to accomplish work regardless of physical location. Microsoft® Office Communications Server 2007 represents the latest opportunity for Microsoft IT to empower users by removing communication barriers, while using existing communications systems and tools. Office Communications Server 2007 includes user features such as instant messaging (IM), presence, Web, audio, and video conferencing, and enables Microsoft IT to provide a security-enhanced platform for communication based on voice over IP (VoIP) technology.

Microsoft IT has incorporated the existing TCP/IP and time-division multiplexing (TDM) infrastructure with Office Communications Server 2007 to provide the best quality of service to all users, maintain network security across the enterprise environment, and validate the product as an early adopter. To successfully implement Office Communications Server 2007, Microsoft IT considered typical planning and design aspects such as coexistence, topology, architecture, and server sizing, in addition to Microsoft-specific needs such as user selection criteria.

This technical white paper covers the approach Microsoft IT took when designing the communications infrastructure, the specific design decisions made, and the deployment strategy and steps used to roll out Office Communications Server 2007 in the production environment. Starting from an overview of the Microsoft IT communications infrastructure, this white paper discusses the business and technical design factors relevant for Microsoft IT, the benefits that the chosen design provides to Microsoft, and the deployment details. This white paper also presents lessons learned and best practices for designing and deploying Office Communications Server 2007. Office Communications Server 2007 provides the option to use VoIP technologies and connects with e-mail messaging components such as Microsoft Exchange unified messaging servers. Additional IT Showcase documents to be released later will provide in-depth coverage of various topics mentioned in this paper, including TDM-based telephony integration and unified messaging.

This paper contains information for business and technical decision makers who are considering deploying Office Communications Server 2007. This paper assumes that the audience is already familiar with the concepts of TCP/IP networks, Windows Server® 2003, the Active Directory® directory service, and VoIP technologies. A high-level understanding of the features and technologies included in Office Communications Server 2007 is also helpful. Detailed product information is available in the Microsoft TechNet Library at http://technet.microsoft.com/en-us/library/bb880155.aspx.

Note: For security reasons, the sample names of forests, domains, internal resources, organizations, and internally developed security file names used in this paper do not represent real resource names used within Microsoft and are for illustration purposes only.

Communications Infrastructure

Prior to deploying Office Communications Server 2007, Microsoft IT managed a centralized communications infrastructure that supported all of the global sites. The infrastructure relied on Microsoft Office Live Communications Server 2005 with Service Pack 1 (SP1) as the platform to provide PC-to-phone functionality, presence, IM, and single-party audio and video. Because Microsoft IT previously implemented Office Live Communications Server 2005 with SP1, the group could build upon the existing infrastructure and take advantage of previous design and deployment decisions. For more information about how Microsoft IT designed and deployed Office Live Communications Server 2005 with SP1, refer to http://www.microsoft.com/downloads/details.aspx?familyid=61e90c28-3840-4c89-9b19-2814a6a62aaf&displaylang=en.

The deployment options and design decisions that Microsoft IT made for the transition to Office Communications Server 2007 depended on not only the existing network infrastructure and Active Directory environment, but also on the telephony infrastructure. Additionally, because Office Communications Server 2007 integrates with Microsoft Exchange Server 2007 unified messaging, Microsoft IT also considered the e-mail messaging infrastructure.

To understand the factors that contributed to the design decisions that Microsoft IT made, it is important to understand the previous environment and infrastructure. Figure 1 shows the previous TCP/IP network, telephony infrastructure, and messaging environment.

Cc297283.image001(en-us,TechNet.10).gif

Figure 1. Network and telephony infrastructure before Office Communications Server 2007

As Figure 1 shows, Microsoft IT deployed Live Communications Server 2005 with SP1 in the IT perimeter network and in the Redmond Active Directory site that is part of the corporate forest. The perimeter network contained access proxies for employee and federated partner use, and an external director, whereas the corporate forest contained a load-balanced front-end server pool, a two-node Microsoft SQL Server® cluster, an application server, a bridgehead server, two internal directors, and archiving agent servers with an associated database server. Microsoft IT implemented remote call control (RCC) by using a third-party computer-telephony integration/private branch exchange (CTI/PBX) and public switched telephone network (PSTN) gateways between the IP network and TDM PBX.

Other aspects of the existing environment that Microsoft IT considered for the deployment of Office Communications Server 2007 included:

  • Geographic location This consideration relates both to data centers that house servers and to locations enabled for communications services. Microsoft IT has three major data centers, one each in Singapore, Redmond, and Dublin, and supports facilities in 98 countries. To accommodate the remote data centers and provide optimal performance to users worldwide, Microsoft IT chose to geographically disperse the topology.
  • Wide area network (WAN) connectivity and bandwidth Microsoft IT relies on Gigabit Ethernet and Synchronous Optical Network (SONET) links in the Redmond location and WAN links of 155 megabits per second (Mbps) to the three major data centers. For telephony connectivity, Microsoft IT uses multiple T1 lines with channel-associated signaling (CAS)/Q.SIG signaling integration.

Opportunities with Office Communications Server 2007

As part of its mission to help product groups deliver software that is proven in a production environment, Microsoft IT examines its IT environment from the viewpoint of a critical customer, including performing gap analysis and considering what must be in place to design, deploy, and operate an IT environment. For Microsoft IT, transitioning to Office Communications Server 2007 from Live Communications Server 2005 with SP1 provided an opportunity to take advantage of new features, such as the Microsoft RoundTable communications and archival system and Enterprise Voice. Together with the unified messaging capabilities in Exchange Server 2007, Office Communications Server 2007 gives employees communications tools for voice, IM, and conferencing in a tightly integrated way by using familiar clients such as Microsoft Office Outlook® 2007. IT administrators and operators also benefit from improved administrative tools and Active Directory based configuration objects.

"Office Communications Server 2007 gives Microsoft IT the ability to offer users more robust and flexible communication tools without drastic changes to the existing infrastructure."

Mark Achzenick
Group Manager
Microsoft Corporation

Microsoft IT was looking forward to taking advantage of improvements in Office Communications Server 2007, including:

  • Streamline communications across IM, presence, audio/video (A/V) conferencing, and telephony Office Communications Server 2007 brings a more consistent user experience across a variety of communication modes, with one interface for managing IM through conferencing and telephone calls.
  • Improve conferencing options with the on-premise server Audio conferences at Microsoft typically include three to four participants who can dial in through a third-party provider bridge. Microsoft IT can decrease costs by using the built-in IM, presence, and conferencing capabilities of Office Communications Server 2007 over the IP network. In addition, new multiparty IM features enable users to add new participants to an existing IM conversation and to escalate a conversation to an on-premise, audio/video Web conference.
  • Increase return on investment (ROI) for conferencing Microsoft used the internal meeting services of Office Communications Server 2007 and replaced the existing on-premise online meeting services, which resulted in a higher ROI.
  • Transition to Microsoft RoundTable RoundTable is an all-in-one video-conferencing phone device that can capture and broadcast synchronized audio and video of a 360-degree panoramic view of everyone in a meeting. This capability is important for Microsoft because it breaks down communication barriers caused by location and provides a richer meeting communication experience to participants.
  • Eliminate single points of failure Microsoft IT recognized the increasing importance of Office Communications Server 2007 as part of the overall unified communications infrastructure, and wanted to ensure high availability by eliminating single points of failure. The architecture of Office Communications Server 2007 made this possible at multiple levels, such as many data centers housing servers, multiple servers for each server role, redundant telephony links, hardware and software load balancing, and multi-node SQL Server cluster integration.
  • Increase user mobility Office Communications Server 2007 enables specified Microsoft users to make VoIP calls from anywhere in the world with a sufficient Internet network infrastructure to support audio and video calls. Office Communications Server 2007 supports several clients for mobile users, including Microsoft Office Communicator Mobile, Communicator Web Access, and Microsoft Office Communicator 2007. These clients provide the rich IM and presence functionality that mobile users need for flexible communication on the road and at home.
  • Enhance security Office Communications Server 2007 not only provides enhanced security by design because it was developed in compliance with the Trustworthy Computing Security Development Lifecycle (SDL), but also enhances security in the following areas:
    • Network communication Most network communications in Office Communications Server 2007 are encrypted by default. All servers use certificates and use Kerberos authentication, Transport Layer Security (TLS), Secure Real-Time Transport Protocol (SRTP), and other industry-standard encryption techniques to help protect Office Communications Server 2007 data on the network. In addition, role-based setup makes it possible to deploy Office Communications Server so that only the services, and the permissions related to those services, are installed as appropriate on each server role.
    • Server authentication Office Communications Server 2007 relies on certificates for server authentication and to establish a chain of trust between clients and servers and among the different server roles. The Windows Server 2003 public key infrastructure (PKI) provides the infrastructure for establishing and validating this chain of trust.
  • Improve logging and compliance Microsoft IT can use the included call detail record (CDR) functionality to record when users log on and log off, as well as to record details about a meeting and specific IM conversations. CDRs enable Microsoft IT to report on call usage and thereby learn more about how the system is being used. Even though Office Communications Server 2007 offers the capability to record the details of all conversation, text, and audio and data collaboration, Microsoft has implemented a subset of these details in accordance with directions from the Microsoft legal team. Microsoft IT has deployed the Office Communicator 2007 client in a way that enables the user to choose to either store or not store conversations based on corporate legal guidelines.

Note: For a comprehensive list of Office Communications Server 2007 documents that are available for download, refer to the Microsoft Office Communications Server 2007 Documentation Roadmap at http://www.microsoft.com/downloads/details.aspx?familyid=3fe6fee9-2342-4726-af48-11b57d1fc9e7&displaylang=en.

To learn about features and functionality in Office Communicator 2007, refer to the Microsoft Office Communicator 2007 What's New Guide at http://www.microsoft.com/downloads/details.aspx?familyid=39B9A6D4-5862-4A01-A2A6-AC193E77AED7&displaylang=en.

Design Challenges and Considerations

In considering the design necessary to take advantage of the improvement opportunities, Microsoft IT began with broad considerations that shaped the overall design such as topology, and moved on to more focused considerations, such as specific security settings. Microsoft IT also adopted several overall goals to guide the group in designing the infrastructure. These goals provided a framework in which to make decisions. For example, Microsoft IT set the goal early on to increase the service reliability and availability by eliminating single points of failure In order to eliminate single points of failure. Accordingly, Microsoft IT designed a topology with multiple locations, fault-tolerant server instances, and load balancing at the front-end and back-end servers.

"The concept of server roles and server pools as logical building blocks in our designs really simplified planning considerations for scalability, sizing, and management. For example, we were able to create uniform, load-balanced server pools and Edge server designs, and use them across all data centers in the internal and perimeter networks. Perhaps best of all, Office Communications Server 2007 supports these logical design building blocks out of the box through server roles."

Indranil Banerjee
Senior Program Manager
Microsoft Corporation

Specifically, Microsoft IT considered the following aspects when designing an infrastructure to support Office Communications Server 2007:

  • Consolidated and expanded server pool topologies The Office Communications Server 2007 architecture includes the concept of server pools that reside in the internal network and consist of front-end, Web component, and conferencing servers with a SQL back end. Microsoft IT, as part of its product validation directive, had to deploy both an expanded server pool with multiple physical servers hosting A/V and Web components, and a consolidated server pool where each server hosted all conferencing components.

    Note: Although Microsoft IT deployed an expanded server pool in the Redmond data center with consolidated pools in the Singapore and Dublin data centers at release to manufacturing (RTM), the group later transitioned the Redmond location to use two consolidated server pools for better supportability.

  • Distributed or centralized management Microsoft IT monitors and operates the IT infrastructure centrally from the Redmond location, and it uses regional IT teams to handle specific operational and deployment tasks. Accordingly, Microsoft IT did not change to decentralized management with Office Communications Server 2007; it continued to manage the entire IT infrastructure centrally.
  • Coexistence and migration phases As part of its pre-release verification tasks, Microsoft IT supported multiple pre-release builds of Office Communications Server 2007, as well as Live Communications Server 2005 operating simultaneously.
  • Site selection When considering the sites in the topology to house Office Communications Server 2007, Microsoft IT initially kept two aspects in mind. First, Microsoft IT considered the user needs. Locations with the most users received higher priority. Second, Microsoft IT considered the existing infrastructure. It is more straightforward and faster to deploy Office Communications Server 2007 in sites that have existing VoIP gateways; therefore, those sites received higher priority.
  • Fault tolerance and eliminating single points of failure Microsoft IT wanted to eliminate all single points of failure for the deployment of Office Communications Server 2007, from the broadest applicable scope to the narrowest. For example, Microsoft IT eliminated single points of failure at the data-center level by deploying servers in multiple sites: at the connectivity level by using redundant WAN, local area network (LAN), and PSTN links; at the server farm level by using multiple load-balanced servers and multi-node clusters; and at the server and storage level by using design elements such as redundant power supplies and RAID arrays.
  • Load balancing Microsoft IT considered many types of server load. For example, Microsoft IT analyzed the typical user traffic at each location to ensure that PSTN and WAN connections could support the conferencing load. Microsoft IT also considered server load for front-end and back-end servers to distribute the load. By using hardware load balancing for server pools and round-robin load balancing for Edge servers, Microsoft developed a load-balanced design.
  • Flexibility and scalability With server roles, Microsoft IT can readily scale up both individual servers by changing server designs and site capacity by adding new servers. For example, to handle increased federated partner traffic, Microsoft IT can deploy additional Edge servers in the perimeter network. Using multiple sites also increases disaster recovery flexibility because of additional site failover choices.
  • Server sizing Microsoft IT combined sizing recommendations from the product group with previous experience in sizing servers and preliminary testing to arrive at the initial server designs for each role. In the end, Microsoft IT arrived at design specifications close to what the product group recommended. (The "Server Sizing" section later in this paper provides more information about server sizing, including the hardware specifications for each role).
  • Remote access Office Communications Server 2007 introduced the A/V Edge server for media relay, which enables remote users to access services from outside the corporate network without connecting through a virtual private network (VPN). Yet, for Microsoft IT to make use of this server role, it must ensure that security is not compromised. Microsoft IT cleared the server role and specific configurations with the corporate security group, which audited the server for security vulnerabilities before deployment.
  • Security Microsoft IT considered security at many parts of the design. At the topology level, Internet-facing front-end Edge servers reside in a perimeter network, which helps to protect internal resources from being compromised. Office Communications Server 2007 comes with encryption capabilities, such as Mutual Transport Layer Security (MTLS), for traffic. Microsoft IT considered the appropriate encryption method for functions such as IM, Web conferences, audio, and external access. Microsoft IT configured both internal certification authorities (CAs) and external or public CAs for data encryption.
  • Disaster recovery Even though Microsoft IT eliminated single points of failure, it still considered disaster recovery plans. In the worst-case scenario of data-center failure, Microsoft IT can recover by using other data centers to handle the load. Additionally, Microsoft IT created comprehensive documentation about the detailed design decisions in addition to deployment checklists, which enable engineers to repeat configuration and setup steps in case of catastrophic failure.
  • Call authorization and routing As part of its configuration considerations; Microsoft IT examined the dependencies of providing service features to users. Specifically, Microsoft IT noted that to use PC-to-phone functionality, users must be enabled for unified communications.
  • Application compatibility Microsoft IT needed to ensure that any third-party application running on Live Communications Server 2005 would be compatible with Office Communications Server 2007. For example, Microsoft IT needed to ensure that Remote Call Control middleware running on Live Communications Server 2005 with SP1 was compatible with Office Communications Server 2007 to continue to deliver those services to the user base in Office Communications Server 2007.

Environment Topology

Before even considering the specific number of servers, their placement, or their sizing, Microsoft IT encountered design considerations that involved meeting its commitment to the product group of verifying multiple topology options of the product. There are two areas where Microsoft IT made topology decisions: in the perimeter network and with the server pool in the internal network. The perimeter network houses A/V, Web Conferencing, and Access Edge servers, whereas the internal network houses the front-end and back-end servers for conferencing, IM, presence, storage, and archiving.

In pursuing the goal to eliminate single points of failure and provide high availability and scalability, Microsoft IT decided to deploy two sets of load-balanced servers, one set for the Access and Web Conferencing roles, and the other for the A/V Edge role. Figure 2 illustrates the Edge role server topologies. In all the topologies for the perimeter network, servers running Microsoft Internet Security and Acceleration (ISA) Server handle the HTTP and HTTPS traffic between external participants and the internal server pools.

Cc297283.image002(en-us,TechNet.10).gif

Figure 2. Edge server role consolidated and expanded server pool topologies

In the perimeter network, it is possible to deploy one or multiple roles on each physical server; the specific mix depends on the site topology and desired scalability. For example, although Microsoft IT could have dedicated one or more physical servers for each server role, this was not cost-effective because in the Microsoft IT environment, the Web Conferencing and Access Edge roles typically experience less load than the A/V Edge role that relays media. Consequently, Microsoft decided to deploy the Web Conferencing and Access Edge roles together and use dedicated servers for the A/V Edge role.

The roles in Office Communications Server 2007 enable the deployment of a consolidated and expanded topology for the server pools. A consolidated topology helps simplify setup and management, while still providing high availability and failure recovery by running the Focus, Focus Factory, Conferencing Server Factory, and conferencing server features on all front-end servers. In an expanded configuration, the A/V Conferencing Server and Web Conferencing server roles are distributed and run on separate servers. This configuration improves scalability by enabling Microsoft IT to add servers of a specific type when necessary. An expanded server pool topology enables Microsoft IT to scale conferencing servers independently of the presence, signaling, and conference control elements that run on front-end servers. Figure 3 illustrates consolidated and expanded server pool topologies.

Cc297283.image003(en-us,TechNet.10).gif

Figure 3. Consolidated and expanded server pool topologies at Microsoft

Microsoft IT decided to deploy the expanded server pool topology in the Redmond data center that serves approximately 80,000 users in North and South America. The team also decided to initially deploy a consolidated topology for the two other data centers in Singapore and Dublin, with the goal of transitioning all sites to a consolidated server pool topology after completing product verification. You can find more details about the topology and Microsoft IT's reasons for making specific deployment decisions in the section titled "Global Topology".

Note: For more information about selecting an appropriate topology for an Office Communications Server 2007 deployment, refer to the Office Communications Server 2007 Planning Guide at http://www.microsoft.com/downloads/details.aspx?familyid=723347C6-FA1F-44D8-A7FA-8974C3B596F4&displaylang=en.

Distributed or Centralized Management

Before deploying Office Communications Server 2007, Microsoft IT followed a centralized model for design, deployment, and operations. This entailed using dedicated teams that handled a specific product, and shared service teams that performed product-independent processes, such as front-line server monitoring. For Office Communications Server 2007 specifically, Microsoft IT saw no business reason to use a distributed management model where each site managed its own design and deployment processes. Just the opposite, the Office Communications Server team in Microsoft IT handles all the design, deployment, and operations aspects. When necessary, the team uses third-party providers and local, on-site IT teams for some unified communications related deployment aspects, such as VoIP gateway installation and telephony integration.

Migration Phases

Deploying all new services at Microsoft occurs in phases; this is a Microsoft IT best practice because it enables fine-tuning of the design, minimizes user impact if changes must be made, and mitigates the risk of deploying new services. Microsoft IT identified two levels at which it could phase in Office Communications Server deployment: the data-center level and the client level. Microsoft IT planned to deploy the hardware in the three data centers simultaneously in order to provide users with the new services worldwide. Each data center handles traffic for specific geographic locations. For example, the Singapore data center handles traffic from Asia and Australia, whereas the Dublin data center handles traffic from Europe and Africa.

After deploying the hardware and configuring Office Communications Server 2007, Microsoft IT began a client-level phased-in migration. Microsoft IT accomplished this by first identifying multiple locations homed to each data center, and then identifying groups of users in each location to migrate. To ensure reasonable adoption rates, Microsoft IT targeted one department or building at each location at a time. Users from specific departments or buildings formed the initial user pool. At the client level with the user pool, Microsoft IT pursued the following methodology:

  1. Phase I Migrate the users in each department or building who are part of Microsoft IT to minimize risk and impact.
  2. Phase II Expand the pilot to additional users picked randomly from the same locations.
  3. Phase III Expand the pilot to more locations homed at each data center.

This approach reduced the risk with regard to the installation of a new client and to how the client would work in a production environment. Most importantly, the gradual ramp-up approach provided a gating factor in terms of support. Microsoft IT was able to identify issues with smaller batches of users, fix those issues, and then move forward with the deployment to larger batches of users.

Site Selection

As already mentioned, Microsoft IT considered the number of users and existing infrastructure when choosing the initial sites to be enabled for unified communications with Office Communications Server 2007. Yet, these were not the only considerations. For example, Microsoft IT also made sure that the site had adequate bandwidth to support the additional traffic, both peer to peer, and client to mediation server. On the telephony side, the PBX needed to have Q.SIG T1/E1 ports for connecting to the VoIP gateway. Additionally, Microsoft IT made sure that all potential locations were cleared of regulatory and homologation hurdles for gateway and client devices.

Note: In some countries, use of VoIP is illegal. Enterprises considering a deployment to a foreign country should ensure that the country allows VoIP. They should also ensure that the local governments in those countries have qualified and approved devices that run Office Communications Server 2007.

Load Balancing

Microsoft IT wanted to ensure that users experienced a high quality of service with Office Communications Server 2007. Although sizing servers properly, distributing the topology to multiple data centers, and ensuring adequate bandwidth provided a sound foundation for a high quality of service, using load balancing at multiple points gave Microsoft IT yet another level of control for optimization. Microsoft IT decided to use both hardware load balancers and Domain Name System (DNS) round robin in its design. Specifically, Microsoft IT uses the following strategies with these server roles:

  • Server pool Microsoft IT uses a third-party hardware load balancer for the front-end servers in the server pool. This is the case with both the expanded and consolidated configurations. Using two server pools in each data center in the consolidated configuration provides load balancing in addition to redundancy.
  • A/V Edge servers Microsoft IT uses DNS round robin to balance the media traffic that the A/V Edge servers handle. Microsoft IT chose to use round robin (an unsupported configuration) initially instead of a hardware load balancer for expediency in order to deploy A/V Edge servers without the need for the additional training and support requirements associated with a hardware load balancer. In the future, Microsoft IT plans to implement a hardware load balancer for A/V Edge servers.
  • Director servers Microsoft IT uses a third-party hardware load balancer for servers that host the Director role.
  • Access and Web Conferencing Edge servers The servers that host the Access Edge and Web Conferencing Edge roles are load-balanced with a third-party hardware load balancer.

Server Sizing

Office Communications Server 2007 introduced several new server roles, such as the A/V Conferencing server, Web Conferencing server, and IM Conferencing server, to support the new on-premise Web conferencing and Enterprise Voice functionality that Microsoft IT could include in its infrastructure. Specifically, the following roles are available:

  • Access Edge server Formerly known as the Access Proxy, this server handles all Session Initiation Protocol (SIP) traffic that crosses the corporate firewall. The Access Edge server is required for all external user scenarios, including conferencing, remote user access, federation, and public IM connectivity.
  • Web Conferencing Edge server This server proxies Persistent Shared Object Model (PSOM, a Microsoft Office Live Meeting protocol) traffic between the Web Conferencing server and external clients. The Web Conferencing Edge server must authorize external conference traffic before it is forwarded to the Web Conferencing server. The Web Conferencing Edge server requires external clients to use TLS connections and obtain a conference session key.
  • A/V Edge server This server provides a single trusted connection point through which audio and video traffic enters and exits a network. With an A/V Edge server, users can add audio and video data to meetings with external participants, and share audio and video directly with an external user (point-to-point). The A/V Edge server also handles audio for Enterprise Voice.
  • Mediation server This server acts as a bridge between the Enterprise Voice components and the media gateway, IP-PSTN gateway (a partner product). A Mediation server will route a PSTN call to the appropriate Enterprise Voice client and vice versa.
  • Front-end server This server houses instant messaging, presence, telephony, conferencing servers, and all essential user services operations.
  • Conferencing server Supporting multiparty conferences requires a new server role known as a conferencing server (also known as a multipoint control unit or MCU). A conferencing server is a pluggable component that manages one or more media types. Office Communications Server 2007 includes four conferencing servers and the extensible architecture for adding more:
    • IM Conferencing server Enables group IM by relaying IM traffic between all participants. When a third participant is added to a peer-to-peer IM conversation, the initiating client invites the IM Conferencing server to the conversation. From that point, all messages between the participants are routed through the IM Conferencing server. The IM Conferencing server is an integral part of the front-end server and cannot be installed on a separate computer.
    • Telephony Conferencing server Responsible for audio conferencing provider (ACP) integration. Supports both dial-out and dial-in, in addition to standard third-party call control features such as mute and eject. The Telephony Conferencing server does not support mixing VoIP and PSTN in the same call. Connecting dial-out to PSTN endpoints requires a Mediation server, as described later in this paper. The Telephony Conferencing server is an integral part of the front-end server and cannot be installed on a separate computer.
    • Web Conferencing server Manages conference data collaboration, including native support for Microsoft Office PowerPoint® presentations, Microsoft Office document sharing, white boarding, application sharing, polling, Q&A, meeting compliance logging, annotations, handouts, and various multimedia formats. The Web Conferencing server uses PSOM for uploading slides to a meeting. The Web Conferencing server can reside either on the front-end server (Standard Edition and Enterprise Edition Consolidated Pool) or on a separate physical computer (Enterprise Edition Expanded Pool).
    • A/V Conferencing server Provides multiparty IP audio and video mixing and relaying, including RoundTable, by using industry-standard Real-Time Transport Protocol (RTP) and Real-Time Transport Control Protocol (RTCP). The A/V Conferencing server can reside either on the front-end server (Standard Edition and Enterprise Edition Consolidated Pool) or on a separate physical computer (Enterprise Edition Expanded Pool).
  • Web Components server This server enables users to access meeting presentations and other content from Web conferences; download Address Book server files to provide Office Communicator with global address list information; expand membership in distribution groups; and obtain other data that the Web Conferencing server uses.
  • Director The director server is a server role that can be deployed in standard or enterprise deployments with remote or federated users who access internal servers. The Director role offloads authentication tasks from front-end servers by running only authentication services. In this way, front-end servers can focus resources on Web conferencing, A/V conferencing, and IIS services. Microsoft IT decided to use one Director server running Microsoft Office Communications Server 2007 Standard Edition in the Redmond data center.
  • Microsoft Office Communicator Web Access Office Communicator Web Access provides a server scale web client for Office Communications Server that targets the following three platforms: security-hardened computers running the Windows® operating system with no permissions to install Office Communicator 2007; computers not running a Windows operating system that cannot install Office Communicator 2007; and mobile devices that have Web browsing capability and are compatible with Asynchronous JavaScript and XML (Ajax) technology or custom clients.

Both Microsoft IT and the Office Communications Server 2007 product group performed repeated load tests on all the server roles by using early code versions of the software to determine how many users could be supported and the hardware requirements for each server role. They used these numbers to design the initial architecture. To ensure that Microsoft IT was able to meet the high capacity demands associated with supporting a growing number of users, it used dedicated servers, as shown in Table 1.

Table 1. Server Specifications for Server Roles in Office Communications Server

Role

Consolidated

Expanded

Front End

4 2.33-gigahertz (GHz) processor, plus 8 gigabytes (GB) of RAM

2 2.33-GHz processor, plus 4 GB of RAM

File Server

4 2.33-GHz processor, plus 8 GB of RAM

4 2.33-GHz processor, plus 4 GB of RAM

Communicator Web Access

Not applicableחsingle Communicator Web Access server in Redmond

2 2.33-GHz processor, plus 4 GB of RAM

A/V Conferencing

Not applicableחconsolidated

4 2.33-GHz processor, plus 8 GB of RAM

Data (IM, Web, Telephony) Conferencing

Not applicableחconsolidated

4 2.33-GHz processor, plus 8 GB of RAM

Access Edge

2 2.33-GHz processor, plus 8 GB of RAM

2 2.33-GHz processor, plus 8 GB of RAM

A/V Edge

2 2.33-GHz processor, plus 8 GB of RAM

2 2.33-GHz processor, plus 8 GB of RAM

SQL

4 2.33-GHz processor, plus 16 of GB RAM

4 2.33-GHz processor, plus 16 GB of RAM

Archiving

4 2.33-GHz processor, plus 8 GB of RAM

4 2.33-GHz processor, plus 8 GB of RAM

Director

2 2.33-GHz processor, plus 4 GB of RAM

2 2.33-GHz processor, plus 4 GB of RAM

Mediation

4 2.33-GHz processor, plus 8 GB of RAM

4 2.33-GHz processor, plus 8 GB of RAM

Web Components

Not applicableחconsolidated

2 2.33-GHz processor, plus 8 GB of RAM

After arriving at the initial processor and memory configuration for each server type, Microsoft IT needed to determine the number of servers required at each data center to support the users. To determine the number of servers at each location, Microsoft IT had to know the maximum number of concurrent users each server could support with no loss in service quality, and the total number of users homed to each data center.

Although determining the number of users homed to a data center was straightforward (Microsoft IT counted the total users at each site homed to a data center and calculated the sum, which came to 24,118 for Dublin, 24,649 for Singapore, and 86,587 for Redmond), determining the maximum concurrent users proved more of a challenge because that depends on the usage patterns. For example, one usage pattern may be that federated partners rarely hold videoconferences, and all internal users use the Office Communicator 2007 client. Another pattern may have users using primarily IM and no audio or video. Therefore, Microsoft IT made assumptions about typical usage based on statistics gathered from operating Live Communications Server 2005 with SP1. Among other things, Microsoft IT considered these usage pattern factors:

  • Client choice As mentioned later, Office Communications Server supports many devices, including Office Communicator 2007, Communicator Web Access, and Office Communicator Phone Edition (VoIP) endpoints. Microsoft IT used these three clients because they are the most commonly used types. Microsoft IT assumed that at any time at any location, 100 percent of the users were connected through Office Communicator 2007, 5 percent were connected through a VoIP endpoint device, 4 percent were connected through Office Communicator 2005, and 1 percent were connected through Communicator Web Access.
  • Active users Microsoft IT estimated that approximately 65 percent of users homed to a specific data center were actively using Office Communications Server 2007. To make room for growth, Microsoft IT added 10 percent to the active user estimates, arriving at 18,089 for the Dublin data center, 18,487 for the Singapore data center, and 65,940 for the Redmond data center.
  • Server role connections The usage patterns differ depending on the server role to which a client is connected. For example, whereas an Office Communicator 2007 client may be connected to an Access Proxy Edge role server 5 percent of the time, an Office Communicator 2005 client may be connected to an Access Proxy Edge role server 100 percent of the time. Table 2 shows the usage model Microsoft IT used for the consolidated and expanded topologies per role and client type.

Table 2. Usage Model per Role and Client Type

Component

Office Communicator 2007

VoIP endpoint

Office Communicator 2005

Communicator Web Access

Expanded topology

Active users

100%

5%

4%

1%

Front-end

100%

100%

100%

100%

Back-end

100%

100%

100%

100%

Communicator Web Access

0%

0%

0%

100%

Archiving Database

100%

100%

100%

100%

File Server

100%

100%

100%

100%

Quality of Experience (QoE)

0%

0%

0%

0%

Web Conferencing

0%

0%

0%

0%

A/V MCU

4%

4%

100%

0%

Data MCU

0%

0%

100%

0%

A/V Edge

1%

4%

100%

0%

Web Edge

0%

0%

100%

0%

Access Proxy Edge

5%

4%

100%

0%

Edge Data Relay

5%

4%

100%

0%

Consolidated topology

Active Users

100%

5%

4%

1%

Front-end + A/V + Data + Web Conferencing

100%

100%

100%

100%

BE

100%

100%

100%

100%

Communicator Web Access

0%

0%

0%

100%

AA

100%

100%

100%

100%

Archiving Database

100%

100%

100%

100%

File Server

100%

100%

100%

100%

QoE

0%

0%

0%

0%

A/V Edge

1%

4%

100%

0%

Web Edge

0%

0%

100%

0%

Access Edge

5%

4%

100%

0%

After Microsoft IT created a usage model and knew the total number of active users homed to each database, it was able to calculate the number of servers required. To do this, Microsoft IT first obtained recommendations from the product group about the number of simultaneous connections each server type could accept without performance degradation, and then calculated the typical number of connections based on the usage model. This information provided the required quantity of servers for each server type in both expanded and consolidated topologies.

For example, for the Redmond data center, Microsoft IT calculated the number of simultaneous connections for servers by first calculating the typical number of clients connected for each client type (64,940*100%=64,940; 65,000*5%=3,247; 65,000*4%=2,598; 65,000*1%=649), and then calculating the typical number of clients connected for the specific server role (for front-end servers, because the usage is 100 percent for all four client types, this would be 64,940+3247+2,598+649=70,785). Because front-end servers in the expanded topology can handle up to 10,000 concurrent sessions, it takes 71,500/10,000=7.15 servers to handle the load. Correspondingly, Microsoft IT planned to deploy eight front-end servers in the Redmond data center. Table 3 shows the maximum number of concurrent sessions per server role and the number of sessions per data center.

Table 3. Server Specification for All Server Roles in Office Communications Server

Topology or category

Role

Maximum concurrent sessions per server

Dublin sessions

Singapore sessions

Redmond sessions

Expanded

Front End server

10,000

Not applicable

Not applicable

70,785

Back End SQL server

50,000

Not applicable

Not applicable

70,785

Web Component server

25,000

Not applicable

Not applicable

0

A/V Conferencing server

525

Not applicable

Not applicable

5,325

Data (IM, Web, Telephony) Conferencing server

625

Not applicable

Not applicable

2,598

Consolidated

Back End SQL server

25,000

19,897

20,335

Not applicable

Front End, Web Component, A/V Conferencing, and Data Conferencing server

5,000

19,897

20,335

Not applicable

Standard Edition

Front End, Web Component, A/V Conferencing, and Data Conferencing with SQL Server Desktop Engine or Microsoft SQL Server 2005 Express Edition server

5,000

Not applicable

Not applicable

Not applicable

Edge

Stand-Alone Access Edge

10,000

1,664

1,701

5,975

Stand-Alone Web Conferencing Edge

3,000

724

739

2,598

Stand-Alone AV Edge

1,500

941

961

3,377

Collocated Access and Web Edge

6,500

Not applicable

Not applicable

5,975

Collocated Access, Web, and A/V Edge

5,250

Not applicable

Not applicable

Not applicable

Other

Archiving Agent

100,000

19,897

20,335

70,785

Archiving Database

100,000

19,897

20,335

70,785

File Server

100,000

19,897

20,335

70,785

Communicator Web Access

Not applicable

181

Not applicable

Not applicable

Director

Not applicable

Not applicable

Not applicable

Not applicable

Mediation

Not applicable

Not applicable

Not applicable

Not applicable

With these calculations of the actual number of servers necessary for each data center, Microsoft IT made adjustments as necessary to achieve load balancing, redundancy, high availability, and other goals. For example, although theoretically two Edge servers in the perimeter network could handle the load for the Singapore data center, Microsoft IT deployed Edge servers load balanced in pairs to provide high availability. Microsoft IT's server design is only one possibility with Office Communications Server 2007 and does not necessarily reflect the capacity limits supported by the product. For more information about design planning, refer to the Office Communications Server 2007 Planning Guide at http://www.microsoft.com/downloads/details.aspx?familyid=723347C6-FA1F-44D8-A7FA-8974C3B596F4&displaylang=en.

Remote Access

The three Edge server roles and the reverse proxy server provide remote capabilities for federated partners and other remote users who participate in audio and video conferences, as explained earlier. These servers communicate by using a variety of protocols, such as RTP, MTLS, SIP, and Hypertext Transfer Protocol over Secure Sockets Layer (HTTPS), through the associated ports. Accordingly, Microsoft IT submitted the required firewall settings to the security team, which approved the open ports. Figure 4 illustrates the ports used for communication with each Edge server role.

Cc297283.image004(en-us,TechNet.10).gif

Figure 4. Firewall open ports in perimeter network

Microsoft IT decided to use the Redmond location as the primary access point for all users and consequently deployed the Access Edge server role along with the Web Conferencing and A/V Edge roles in the Redmond data center. To eliminate the single point of failure for remote access in Redmond, Microsoft IT decided to deploy the Access Edge role in the Dublin location as well, but only as a failover in case the Access Edge servers in Redmond are unavailable. The Communicator 2007 client natively supports retrying all servers returned by DNS if a primary server is unavailable. By specifying the Access Edge servers in Redmond as primary and the Access Edge server in Dublin as secondary, Microsoft IT makes use of the built-in bootstrap functionality in the Communicator 2007 client for failover.

In the Singapore location, Microsoft IT deployed the Web Conferencing and A/V Edge roles. Because the WAN backbone supports transfer speeds of 155 Mbps and higher, Microsoft IT did not encounter latency issues with communication.

Security

Even with having the built-in authentication and encryption capabilities in Office Communications Server 2007, Microsoft IT needed to consider and design the proper strategy for having a security-enhanced environment at all levels. As with other design considerations, Microsoft considered the security aspects systematically by tracking the flow of data and authentication protocols inside and outside the corporate network. Microsoft IT considered the following security aspects in its design:

  • Internet boundary Microsoft IT explicitly defines the Edge servers as trusted communication partners with internal servers by using security certificates, MTLS, and defined server lists in Active Directory. The Director role helps to increase security by authenticating external enterprise users and routing these users to their home pools. Because Access Edge servers do not communicate with Active Directory, using them in combination with Directors helps defend against distributed Internet attacks, where attackers are posing as remote users. Each of the three regional data centers provides its own perimeter network for Edge servers. However, the Redmond data center houses the primary Access Edge servers used for remote access, whereas the Dublin data center houses the failover Access Edge servers.
  • On-premise conferences Using on-premise Web conferences was a key goal for Microsoft IT, yet meeting the goal included security considerations. Office Communications Server 2007 provides built-in safeguards to help reduce risks, such as limited access and control based on participant roles, meeting types with access controls for participant types, conference scheduling restricted to users enabled in Active Directory, and unique passwords for unauthenticated users to join meetings.
  • Enterprise Voice Microsoft IT decided to deploy Mediation servers in separate virtual LANs (VLANs) to help control network traffic and provide better segmentation and QoS capabilities. Microsoft IT also takes advantage of the built-in security by configuring Mediation servers to use MTLS and SRTP for all communications.
  • General network infrastructure and server hardening Microsoft IT hardens all servers as part of general security practices. For example, Microsoft IT hardens the operating system by disabling unused services and enabling communication only on the necessary ports. Microsoft IT uses Internet Protocol security (IPsec) in the network as part of general security measures. To support the required quality of service for communications, Microsoft IT configures Office Communications Server 2007 to use IPsec but not run in required mode.
  • Office Communicator client Microsoft IT uses Group Policy settings to enforce security for clients. For example, Microsoft IT enforces media encryption and encrypted signaling by using TLS.

Active Directory handles user authentication, and Microsoft IT uses both NTLM and Kerberos as authentication protocols. Although Microsoft IT prefers the use of Kerberos, it must use NTLM to support earlier platforms.

All traffic passing through the pool of front-end servers has a Secure Sockets Layer (SSL) connection and a high level of encryption. This includes IM and presence sessions, in addition to Web conferencing traffic. Office Communications Server 2007 requires certificates to be installed on each server that uses MTLS, which is used to handle security for server-to-server communications. This requirement ensures that each server will validate the other server's certificate. For communications between the server and a client, Microsoft IT uses the TLS protocol.

Supported Clients

Microsoft IT considered the client availability with Office Communications Server 2007 and decided to initially support the following clients and devices:

  • Office Communicator 2007 An enhanced version of the client that came with Live Communications Server 2005, Office Communicator 2007 provides IM, group IM, and other enhanced capabilities.
  • Office Communicator Web Access This enables users to access IM and presence information over the Internet via HTTPS.
  • Office Live Meeting This newly available client gives internal and external users access to view and hear the A/V that is associated with a meeting. Users can also view information that is made available through data collaboration, in addition to the presenters' shared application.
  • Conferencing add-in for Office Outlook By using this add-in, users who schedule on-premise meetings will view a user interface that is similar to the one they use when scheduling meetings via Office Outlook.
  • RoundTable This newly introduced conferencing phone provides a 360-degree view of a conference for external attendees. In conjunction with Office Live Meeting Console, this device provides a panoramic view of the conference and an active speaker window.
  • Communicator Mobile This is the client for Windows Mobile® 5.0 software.

Note: For more information about Office Communicator 2007, refer to the Office Communicator 2007 home page at http://office.microsoft.com/en-us/communicator/FX101729051033.aspx.

Office Communications Server 2007 Implementation

"Implementing Office Communications Server 2007 was challenging because it integrates e-mail messaging, the IP network, and the traditional PSTN. Even with proper planning, it is impossible to anticipate all eventualities. We mitigate the risk in part through a pilot deployment to a limited user pool before rolling out services to all users."

Bharath Sankaranarayan
Group Program Manager
Microsoft Corporation

After designing the environment, Microsoft IT considered the implementation challenges in order to systematically deploy the designed infrastructure. Microsoft IT considered the following specific, high-level deployment challenges:

  • Client version control Because Microsoft IT deployed multiple code builds in its rollout phases, users had various client builds. Although the latest builds supported prior client versions, Microsoft IT wanted to collect meaningful statistics from uniform client versions. Microsoft IT dealt with this situation by taking measures to increase user adoption of the latest client version by using Microsoft System Center Configuration Manager 2007 and client version control. Microsoft IT used Configuration Manager first because it was a less problematic way to encourage client version adoption. Configuration Manager enabled Microsoft IT to notify users, who then could voluntarily update their clients. This process resulted in approximately 9,000 users upgrading. For the remaining users, Microsoft IT enforced client version control by using the block list capability of Office Communications Server 2007.
  • Version coexistence Microsoft IT faced another challenge in supporting both the hosted Office Live Meeting service and the on-premise Web conferencing service. Because both versions were available, some users were confused about which tool to use. The user adoption group within Microsoft IT prescribed a message of "choose the best tool for the job" for users who wanted to schedule conferences. For example, the user adoption group recommended that users choose the Office Live Meeting hosted service tool for large conferences, such as e-learning sessions. For small, collaborative meetings, the group recommended the Web conferencing capabilities built into Office Communications Server 2007.
  • SQL Server clustering Microsoft IT ran the Live Communications Server 2005 with SP1 environment with clustered SQL servers by using a storage area network (SAN) that was shared with other services, such as messaging and collaboration and handled by the storage services team within Microsoft IT. The product group recommended continuing using the shared SAN with clustered SQL servers for the expanded topology of Office Communications Server 2007. Microsoft IT initially followed this recommendation and deployed Office Communications Server 2007 with clustered SQL servers in the shared SAN. After deploying clustered SQL servers with a SAN, Microsoft IT monitored performance and determined that the shared SAN did not meet the input/output (I/O) demands for Office Communications Server 2007. To try to meet I/O demands, Microsoft IT made the engineering decision to dedicate spindles in the SAN for Office Communications Server 2007. After monitoring the performance with dedicated spindles, Microsoft IT recognized that even the dedicated spindles did not deliver the required I/O throughput. The next logical step would have been to use a dedicated SAN for Office Communications Server 2007. However, this was cost-prohibitive; therefore, Microsoft IT changed the design to use direct-attached storage (DAS). This change removed SQL clustering and provided the required throughput, yet it did not provide the required high availability for a pool that uses an expanded topology. Therefore, Microsoft IT decided to deploy n+1 consolidated pools in Redmond, each with its own dedicated storage through DAS. A consequence of this decision is the lack of automatic failover; it is manual based on certain operational criteria of moving users from a failed pool to an active pool. At RTM, Microsoft IT did not yet implement the n+1 consolidated pool design in the Singapore and Dublin data centers, but Microsoft IT planned to do so.
  • IPsec The Microsoft infrastructure is currently IPsec enabled; however, using IPsec creates overhead for audio and video traffic, which generates delays in session setup. To provide the needed quality of service, Microsoft IT obtained an exception from the corporate IT security team for all the servers running Office Communications Server 2007 from IPsec required mode so that servers do not force IPsec use. Yet even after obtaining the exception, implementing the exception process was challenging. Microsoft IT implemented a new Group Policy object (GPO) policy to maintain the exception on the required computers running Office Communications Server 2007. Any new servers being added to the environment had to be put in the exception list, followed by security approval.

Deployment Phases

Microsoft IT deployed multiple code builds in phases to the three data centers. It also added users in phases to expand the test pool, adding the most users as the product neared completion. Initially, Microsoft IT decided to deploy to the pre-release environment and the corporate production environment. Because Microsoft IT was not ready to deploy the product to all users, the team decided to support Live Communications Server 2005 in parallel with Office Communications Server 2007 and support both versions.

Deployment Checklists

As part of following best practices and standard deployment and operations processes based on industry frameworks such as Microsoft Operations Framework (MOF) and IT Infrastructure Library (ITIL), Microsoft IT develops checklists for a base implementation. After documenting its design decisions, Microsoft IT proceeded to create a prescribed implementation and configuration document that included a deployment checklist. The document guided Microsoft IT to accomplish the tasks shown in Table 4.

Table 4. Office Communications Server 2007 Deployment Checklist

Task

Details

Perform prerequisite steps for DNS, Active Directory, and SQL

Microsoft IT uses scripts and command-line parameters to install and import initial configurations and prepare the environment for Office Communications Server 2007. For example, to prepare the Active Directory infrastructure, Microsoft IT runs the lcscmd /forest /Action:SchemaPrep, lcscmd /forest /Action:ForestPrep, and lcscmd /forest /Action:DomainPrep commands.

Install certificates

Microsoft IT configures certificates for both IIS and Office Communications Server 2007. The Office Communications Server certificate is used for MTLS session creation. Every server that uses SIP must have this certificate configured. To install the certificate, Microsoft IT engineers open Computer Management, right-click Office Communications Server under Service and Applications, and then use the Certificates menu. The IIS certificate is used for SSL session creation. The Web Components service uses IIS, which requires this certificate to be installed. Microsoft IT configures this on the Default Web Site in IIS.

Deploy consolidated servers

Microsoft IT uses a script to deploy the consolidated server for the Singapore and Dublin data centers. The script includes the configuration parameters similar to setupee.exe /onelinesetup /SKIPSTATECHECK /INSTALL:EE(INSTALLDIR="<directory>"),WEB(INSTALLDIR="<directory>"),AVMCU(INSTALLDIR="<directory>"),DATAMCU(INSTALLDIR="<directory>"). After deploying each consolidated server, Microsoft IT manually activates the Web components, Web conferencing, and A/V conferencing, in addition to changing the TLS listening port to 5065 in the Listener configuration until User Replicator is completed.

Deploy first server in expanded configuration

Microsoft first installs the core SIP messaging server in the Office Communications Server environment. This service must be installed before any other in a pool because all other services depend on this service being operational. To install this service, Microsoft IT runs setupee.exe /onelinesetup /SKIPSTATECHECK /INSTALL:EE(INSTALLDIR= "<directory>").

Deploy Web component services

To deploy the Web Component role, Microsoft IT runs the setupee.exe /onelinesetup /SKIPSTATECHECK /INSTALL:WEB(INSTALLDIR="<directory>") command.

Deploy Web conferencing services

To deploy the Web Conferencing role, Microsoft IT runs the setupee.exe /onelinesetup /SKIPSTATECHECK /INSTALL:DATAMCU(INSTALLDIR="<directory>") command.

Deploy audio/video conferencing services

To deploy the A/V conferencing role, Microsoft IT runs the setupee.exe /onelinesetup /SKIPSTATECHECK /INSTALL:AVMCU(INSTALLDIR="<directory>") command.

Deploy Edge servers

Microsoft IT deploys the Edge server roles on separate servers in each data center. However, only the Redmond data center has the Access Edge role installed, with the Dublin data center acting as failover for Access Edge servers. To install the Access Edge role servers, Microsoft IT runs the setupee.exe /onelinesetup /SKIPSTATECHECK /INSTALL:AP(INSTALLDIR="<directory>") command and then activates it by using the lcscmd /server /action:Activate /Role:AP /components: command.

Deploy Director

Microsoft IT installs the Director role by running the setupse.exe /onelinesetup /SKIPSTATECHECK /INSTALL:SE(INSTALLDIR="<directory>") command.

Deploy archiving

Microsoft IT installs archiving by running the SetupEE.exe /onelinesetup /skipstatecheck /install:ARCH(INSTALLDIR="<directory>") command.

Deploy mediation servers

The mediation servers are used for unified communications telephony. They are paired with a gateway. Microsoft IT installs the Mediation role by running the setupee.exe /onelinesetup /SKIPSTATECHECK /INSTALL:MS(INSTALLDIR="directory>") command.

These deployment details are only a start. Microsoft performs additional tasks such as deploying and configuring load balancing. For a more comprehensive list of deployment tasks, refer to "Overview of Enterprise Edition Deployment" at http://technet.microsoft.com/en-us/library/bb663663.aspx.

Global Topology

By the time Microsoft IT deployed the RTM build, the environment supported 75,000 Office Communicator 2007 and Office Live Meeting users with 15,000 concurrent IM connections, 7,500 Office Live Meeting conferences, and 2,500 active Office Live Meeting organizers logged on per month. Microsoft IT initially deployed an expanded topology in the Redmond data center, as shown in Figure 5. However, in the months following the implementation of the RTM build, Microsoft IT transitioned the Redmond data center to use a consolidated topology as well, as shown in Figure 6.

There were several reasons for transitioning the Redmond data center to use a consolidated topology. As already mentioned, Microsoft IT determined that the shared SAN environment did not provide the necessary I/O throughput, which resulted in a configuration of SQL servers that used DAS without clustering. This represents a single point of failure for each server pool and an unacceptable risk for Microsoft IT. To provide full system redundancy, Microsoft IT decided to use multiple server pools at each site, in addition to spreading out the user load across multiple geographically dispersed data centers.

Another reason reasons for transitioning the Redmond data center to use a consolidated topology is simplification. For Microsoft IT, it is more straightforward to implement, administer, and operate a server topology that is homogenous across all data centers. In analyzing the most common reasons for unavailability, Microsoft IT noticed that external dependencies such as the underlying network, hardware load balancers, Active Directory, and so on were responsible. By reducing the complexity of the environment, Microsoft IT reduces the potential for unavailability caused by external dependencies.

Cc297283.image005(en-us,TechNet.10).gif

Figure 5. Microsoft IT global topology at RTM

Cc297283.image006(en-us,TechNet.10).gif

Figure 6. Microsoft IT global topology after RTM

Note: At RTM in August 2007, Microsoft IT did not yet fully deploy Enterprise Voice to all users throughout the United States, Europe, and Asia. However, Microsoft IT deployed the necessary architecture and server support for Enterprise Voice in preparation for providing users with this functionality sometime in 2008.

Deploying New Clients and Devices

The Office Communications Server 2007 beta release included the deployment of the new Office Communicator and Office Live Meeting clients to 15,000 of the new users. Microsoft IT conducted an adoption campaign to inform the users of the value and benefits of using the new clients. Due to the success of the adoption, Microsoft IT deployed the new clients and devices to 75,000 users through a phased-in approach, in which the next set of users received the new clients and devices only after Microsoft IT resolved any outstanding issues with the existing user pool.

Microsoft IT initially deployed the beta release to 13,000 Microsoft Office Communicator 2005 and Office Live Meeting users. It also deployed 144 RoundTable units. In a three-month period, users scheduled more than 1,300 Office Live Meeting conferences. Subsequent releases would increase the number of enabled users to 48,000, with 244 RoundTable devices and more than 4,800 Office Live Meeting conferences in another seven-week period.

Microsoft IT deployed 25 of the 244 RoundTable devices globally to determine the impact that these devices would have on WAN traffic. After evaluating the results, Microsoft IT made plans to implement several thousand additional RoundTable devices in the months following the RTM release. As already mentioned, Microsoft IT ensured that the devices were approved for use in each respective country before deploying them to users.

Deployment Verification

After installing and configuring Edge servers, the server pool, and additional servers for added features such as archiving, Microsoft IT verifies the functionality and features by using a test account and a pool of test users. By the time Microsoft IT performs a detailed test of the environment, Microsoft IT engineers have already configured the firewalls to permit traffic, and they have configured mediation servers with VoIP gateways to forward data. At this point, Microsoft IT tests every location by using the checklist shown in Table 5. IT also tests client functionality, as shown in the checklist in Table 6 and Table 7. Microsoft IT records the result, and if a test did not pass, provides a reason. For each location, Microsoft IT also tests bi-directional audio capability and call setup time.

Table 5. Testing Checklist for Each Location

Case

Scenario

Pass/ conditionally pass/ fail

Reason

1

Make a call from inside the location to an outside number

2

Make a call from outside the location to an internal number

3

Call voice mail

Table 6. Testing Checklist for Office Communicator, Communicator Web Access, and Communicator Mobile Clients

Case

Scenario

Pass/ conditionally pass/ fail

Reason

1

Logon

2

Address Book Service

Verify presence and IM

3a

Group

3b

Internal Dublin

3c

Internal Singapore

3d

Internal Redmond

3e

Internal Exchange

3f

Remote Dublin

3g

Remote Singapore

3h

Remote Redmond

3i

Remote Exchange

3j

Federation

Table 7. Checklist for Live Meeting Clients

Case

Scenario

Pass/ conditionally pass/ fail

Reason

1

Meet now

2

Internal join

3

Set up audio

4

Set up video

5

Remote join

6

Set up audio

7

Set up video

8

Upload content

9

Share content

10

Download content

11

Verify PSTN audio conferencing integration

Operations and Support

While Microsoft IT was determining the number of employees who would be affected by each deployment, it tried to balance the interests of the Office Communications Server 2007 product group against its own ability to handle Helpdesk calls. The team had to overcome challenges, such as how to handle beta user support when the Microsoft IT Helpdesk typically does not support beta products. It needed a solution for supporting users in the best way possible while assisting the product group with testing and with providing valuable feedback.

Tiered Support

During the deployment of Office Communications Server 2007, Microsoft IT maintained support of Live Communications Server 2005, in addition to supporting Office Communications Server 2007. Microsoft IT offers support in four tiers. Tier 1 answers calls placed to the Helpdesk. If necessary, a caller is routed to Tier 2, where the support staff enters a service request. If the support staff cannot resolve the issue at the Tier 2 level, it forwards the issue to Tier 3, which handles issues that the support materials do not cover. If Tier 3 cannot resolve the issue, it directs the issue to Tier 4, where the problem will remain until an engineer resolves it.

In all, the Live Communications Server 2005 and Office Communications Server 2007 installations required six engineers at the Tier 4 level, including engineers to specifically support earlier telephony, Web telephony, and Web collaboration. Microsoft IT needed four operations staff for Tier 3 support and an additional four support staff for Tier 2 support. Microsoft IT expects this number to decrease after Live Communications Server 2005 is phased out entirely.

When Microsoft IT deployed the RTM build in August 2007, the support teams already had extensive expert knowledge for dealing with issues gathered from years supporting Live Communications Server 2005 and from supporting beta versions during the pilot. Microsoft IT had created enough prescribed resolution steps for Tier 1 support members that over 80 percent of support calls were resolved at the Tier 1 level. Table 8 shows the support tickets related to various clients at RTM and the following two months.

Table 8. Support Ticket Percentage Distribution

Client product

August (RTM)

September

October

Communicator 2005

21%

15%

5%

Communicator 2007

65%

63%

57%

Communicator Mobile

1%

1%

2%

Communicator Web Access

<1%

<1%

<1%

Communications Server Web conferencing

3%

2%

5%

Table 9 shows the primary symptoms and factors for support tickets opened during October. By this time, Microsoft IT experienced a significant decline in support tickets after users had an opportunity to configure and use their clients.

Table 9. Top Primary Symptoms by Product in October, Two Months After RTM

Client product

Primary symptom

Percentage of Total Per Client

Communicator 2005

Authentication failure

42%

Help install/uninstall

32%

Presence/status issue

11%

IM conversation failure

8%

Phone integration failure

2%

Outlook integration issues

1%

Communicator 2007

Help install/uninstall

44%

Phone integration failure

20%

Outlook integration issues

8%

Presence/status issue

8%

IM conversation failure

6%

Authentication failure

6%

PC-to-phone audio failure

2%

Live Meeting add-in failure

2%

Application sharing failure

1%

Communicator Mobile

Authentication failure

39%

Help install/uninstall

35%

Administrative Support

The Microsoft IT team that supports issues related to the installation of Office Communications Server 2007 consists of the same individuals who helped to design and deploy the product. They are also responsible for the administration of Office Communications Server 2007. The team uses scripting not only for deployment but also for ongoing support. IT accomplishes scripting by using Microsoft Visual Basic® Scripting Edition (VBScript) or Windows PowerShell command-line interface.

Microsoft IT also uses Microsoft Identity Integration Server (MIIS) to help provision users. It used MIIS in Live Communications Server 2005 and continued to use MIIS for the Office Communications Server 2007 deployment. MIIS can be used to manage accounts in Active Directory.

To help support clients, an organization can use the Office Communications Server Logger and Snooper tools. These client-side tools provide valuable information to support staff. Office Communications Server Logger runs on the client computer and stores information in log files that support staff can retrieve. Snooper then parses and reads these log files.

Lessons Learned

This section contains some of the lessons that Microsoft IT learned as it deployed Office Communications Server 2007. It also contains suggested practices for organizations that want to deploy Office Communications Server 2007.

  • Start planning early and involve all affected groups To plan the Office Communications Server 2007 deployment, Microsoft IT sought input from several related teams within Microsoft IT and product groups, most notably while implementing network perimeter changes for the data centers. For example, some teams were responsible for DNS changes, and others were responsible for load balancers. Microsoft IT also needed to get design approvals from the network security team. Because multiple teams may be responsible for delivering the system requirements in anticipation of potential delays, organizations that want to deploy Office Communications Server 2007 should start early in their planning and involve every group that the deployment will affect. Additionally, organizations should include the end-user support groups early on to maximize end-user training and prepare the Helpdesk support staff.
  • Obtain executive-level support Microsoft IT obtains executive-level support when establishing a new service at any site. For example, executives send an e-mail message to all staff, announcing their support for Office Communications Server 2007 and explaining the reasoning behind implementing it. Organizations should also seek executive-level support to help drive user adoption.
  • Work closely with telecom engineers when planning the Enterprise Voice implementation Organizations should seek technical support and approval from telecom engineers during the planning and deployment of Enterprise Voice. Microsoft IT had to work closely with gateway and PBX partners during both the planning and deployment processes. Organizations should also dedicate plenty of time to understanding the dial plan codes for each new site. Certain sites, especially sites in other countries, may have different rules regarding how to attain outside lines, or they may use a different phone number length.
  • When deploying to a new site, consider providing temporary on-site support To help resolve issues that occur as new users are enabled, Microsoft IT found that providing temporary on-site support was useful. This support usually involved one support person who would physically reside in the site building.
  • When deploying to clients, ensure that logging and tracing are turned on by default Microsoft IT learned that ensuring that logging and tracing are turned on during deployment of new clients is much easier than discovering that clients needed the services after a problem occurs. By enabling logging and tracing on the client computer, Helpdesk support could gather critical information that it could use to resolve user issues.

Best Practices

The following list represents best practices for planning and implementing an Office Communications Server 2007 deployment:

Planning

  • Conduct a small pilot before rollout.
  • Plan the migration of Edge servers, especially if migrating from Live Communications Server 2005 with SP1 and running the federation service.
  • Create a migration plan for all Office Communicator Mobile and Office Communicator Web Access clients.
  • Estimate bandwidth requirements for deploying RoundTable devices at planned sites.
  • Use tools such as Microsoft Systems Management Server to limit user installation mistakes and increase adoption across the organization.
  • If using previous platforms, set a phase-out date for the previous version to reduce operational support costs. The migration should have a set end date as well.
  • Manage the deployment by using sufficient checklists to ensure that critical tasks are not missed and dependencies are identified.
  • Implement only the functionality that the organization needs.

Server Design and Deployment

  • Document and manage all the certificates for the relevant servers, both internal and external.
  • Use A/V Edge servers for media relay instead of voice over VPN.
  • Use SRTP instead of RTP to help secure audio packets.
  • Use SQL clustering and highly available file shares.
  • Use the service usage model in the product deployment guide to size the infrastructure.
  • Client design and deployment
  • For Office Live Meeting, ensure that the external link in the e-mail invitation is active.
  • Enforce Office Communicator security settings.
  • Ensure that policy limits are in place for remote access users.

Infrastructure Configuration

  • Ensure that the user-provisioning mechanism in MIIS is modified to the requirement for Office Communications Server 2007. Ensure that enhanced presence is enabled for all users.
  • If IPsec is already enabled in the environment, handle all IPsec exceptions before deployment.
  • Ensure availability for remote access by using a secondary access point. As already mentioned, for Microsoft IT, if the primary remote access entry in the United States goes offline, the bootstrap process in the client will discover the next available remote access point in another data center.
  • Ensure that the Dynamic Host Configuration Protocol (DHCP) scopes can support the expanded number and type of IP devices required; ensure that the total number of devices will be supported.
  • Use TLS on every device.
  • Use a Gigabit Ethernet network when possible to avoid future rebuilds. Be aware of autosensing settings and duplex settings at the infrastructure level.
  • Get security approvals for the necessary topology and the required access control lists (ACLs) and ports in the perimeter network.

User Education and Migration

  • Manage the user migration in batches, with smaller batches in the beginning to reduce the risk footprint from a support perspective.
  • Provide a delay of two to three days before starting the user migration from the time that the infrastructure is ready; this will enable the system to stabilize before load is added.
  • Run training sessions to drive adoptions with new features of the product, and primarily with the new Office Live Meeting Console and RoundTable.

Operations

  • Start measuring key operational metrics from CDR to assess system usage against design and to drive usage adoption.
  • Use Office Communications Server 2007 validation wizard deployment to validate servers after the installation to troubleshoot connectivity issues.
  • Use the post-validation tool to verify that the product was installed successfully. Conduct phone number normalization to E.164 format before deploying the product to production.
  • Archive call detail records for auditing and metrics.
  • Use version control to block previous client versions.

Conclusion

Office Communications Server 2007 presents an opportunity for Microsoft IT to add a new and vital communications system to the existing unified communications infrastructure. Microsoft IT previously deployed Exchange Server 2007 for e-mail and unified messaging functionality, which helps standardize e-mail management. Similarly, with Office Communications Server 2007, Microsoft is able to standardize telephony management by moving away from traditional TDM-based devices and networks to VoIP-based devices that use the TCP/IP network.

For Microsoft IT, the infrastructure design for Office Communications Server 2007 had to meet availability, reliability, and scalability requirements while eliminating single points of failure. To achieve such a design, Microsoft IT considered many design factors, such as server sizing, load balancing, and security. Additionally, Microsoft IT planned to deploy Office Communications Server 2007 in phases to minimize deployment risks.

After deploying Office Communications Server 2007, Microsoft IT continues to evaluate the infrastructure and processes for improvement opportunities. For example, the initial deployment used an expanded topology in the Redmond location and consolidated topologies in the other data centers. Microsoft IT later transitioned to a consolidated topology in the Redmond location as well for better supportability and manageability. Microsoft IT designed the server capacity to accommodate an increase of up to 20 percent in users each year with no performance degradation. For added capacity, it is straightforward to add additional servers as needed to the base consolidated topologies.

Microsoft IT sees Office Communications Server 2007 as part of a strategic investment in unified communications technology. With the base TCP/IP and telephony infrastructure in place for many years, unified communications is a logical next step to increase user productivity and equip employees with tools for anywhere, anytime communication. Office Communications Server gives users the ability to communicate on their own terms, while on the road, in the office, or at home. By demonstrating the enterprise readiness of Microsoft unified communications products, Microsoft IT continues to show its commitment both to Microsoft users and to real-world customers.

For More Information

For more information about Microsoft products or services, call the Microsoft Sales Information Center at (800) 426-9400. In Canada, call the Microsoft Canada information Centre at (800) 563-9048. Outside the 50 United States and Canada, please contact your local Microsoft subsidiary. To access information through the World Wide Web, go to:

http://www.microsoft.com

http://www.microsoft.com/technet/itshowcase

http://office.microsoft.com/en-us/communicator/FX101729051033.aspx

Information in this document, including URL and other Internet Web site references, is subject to change without notice. Unless otherwise noted, the companies, organizations, products, domain names, e-mail addresses, logos, people, places, and events depicted in examples herein are fictitious. No association with any real company, organization, product, domain name, e-mail address, logo, person, place, or event is intended or should be inferred. Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation.

Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property.

© 2008 Microsoft Corporation. All rights reserved.

Microsoft, Active Directory, Outlook, PowerPoint, RoundTable, SQL Server, Visual Basic, Windows, Windows Mobile, and Windows Server are trademarks of the Microsoft group of companies.

All other trademarks are property of their respective owners.

© 2009 Microsoft Corporation. All rights reserved. Terms of Use | Trademarks | Privacy Statement
Page view tracker