Deploying Microsoft Office Communications Server 2007
How Microsoft IT Deployed Office Communications Server 2007 and Office Communicator 2007
Technical White Paper
Published: March 11, 2008
|
Situation
|
Solution
|
Benefits
|
Products & Technologies
|
|
As part of its commitment to customers, Microsoft IT validates the latest software
builds and design options for Office Communications Server 2007 and Office
Communicator 2007 in a production environment. With the experiences gained
from deploying and operating the systems, Microsoft IT can offer practical examples
and best practices to customers.
|
Microsoft IT deployed Office Communications Server 2007 and Office Communicator 2007
to employees at Microsoft while the products were still in development. These early
users provided valuable feedback to the product groups and helped to ensure that
the products would meet availability, scalability, and security requirements before
RTM.
|
- Enterprise Voice capabilities
- Enhanced Web collaboration experience and reduced associated costs
- Enhanced group IM and real-time presence
- External Microsoft users can access text, audio, and video communications,
without having to access the corporate network through VPN
- Simplified administration of Office Communications Server and reduced total
cost of ownership
|
- Microsoft Office Communications Server 2007
- Microsoft Office Communicator 2007
- Microsoft Office Live Meeting
- Microsoft Office Communicator Web Access 2007
- Conferencing Add-in for Microsoft Office Outlook
- Microsoft RoundTable
- Microsoft SQL Server
- Windows Server 2003
- Active Directory
|
On This Page
Executive Summary
Communications Infrastructure
Opportunities with Office Communications Server 2007
Design Challenges and Considerations
Office Communications Server 2007 Implementation
Operations and Support
Lessons Learned
Best Practices
Conclusion
For More Information
Executive Summary
Companies of all sizes, and especially Fortune 500 companies with global presence
such as Microsoft, increasingly rely on unified communications systems as a way
to provide workers with flexible workplaces and the latest communication technologies,
and to reduce the costs of operating facilities. The Microsoft Information Technology
(Microsoft IT) group is responsible for ensuring that the 130,000 Microsoft users
worldwide—partners, vendors, employees, and associates—have access
to the best available communication tools to accomplish work regardless of physical
location. Microsoft® Office Communications Server 2007 represents the latest
opportunity for Microsoft IT to empower users by removing communication barriers,
while using existing communications systems and tools. Office Communications Server 2007
includes user features such as instant messaging (IM), presence, Web, audio, and
video conferencing, and enables Microsoft IT to provide a security-enhanced platform
for communication based on voice over IP (VoIP) technology.
Microsoft IT has incorporated the existing TCP/IP and time-division multiplexing
(TDM) infrastructure with Office Communications Server 2007 to provide the
best quality of service to all users, maintain network security across the enterprise
environment, and validate the product as an early adopter. To successfully implement
Office Communications Server 2007, Microsoft IT considered typical planning
and design aspects such as coexistence, topology, architecture, and server sizing,
in addition to Microsoft-specific needs such as user selection criteria.
This technical white paper covers the approach Microsoft IT took when designing
the communications infrastructure, the specific design decisions made, and the deployment
strategy and steps used to roll out Office Communications Server 2007 in the
production environment. Starting from an overview of the Microsoft IT communications
infrastructure, this white paper discusses the business and technical design factors
relevant for Microsoft IT, the benefits that the chosen design provides to Microsoft,
and the deployment details. This white paper also presents lessons learned and best
practices for designing and deploying Office Communications Server 2007. Office
Communications Server 2007 provides the option to use VoIP technologies and
connects with e-mail messaging components such as Microsoft Exchange unified messaging
servers. Additional IT Showcase documents to be released later will provide in-depth
coverage of various topics mentioned in this paper, including TDM-based telephony
integration and unified messaging.
This paper contains information for business and technical decision makers who are
considering deploying Office Communications Server 2007. This paper assumes
that the audience is already familiar with the concepts of TCP/IP networks, Windows
Server® 2003, the Active Directory® directory service, and VoIP technologies.
A high-level understanding of the features and technologies included in Office Communications
Server 2007 is also helpful. Detailed product information is available in the
Microsoft TechNet Library at
http://technet.microsoft.com/en-us/library/bb880155.aspx.
Note: For security reasons, the sample names of forests, domains, internal
resources, organizations, and internally developed security file names used in this
paper do not represent real resource names used within Microsoft and are for illustration
purposes only.
Communications Infrastructure
Prior to deploying Office Communications Server 2007, Microsoft IT managed
a centralized communications infrastructure that supported all of the global sites.
The infrastructure relied on Microsoft Office Live Communications Server 2005
with Service Pack 1 (SP1) as the platform to provide PC-to-phone functionality,
presence, IM, and single-party audio and video. Because Microsoft IT previously
implemented Office Live Communications Server 2005 with SP1, the group could
build upon the existing infrastructure and take advantage of previous design and
deployment decisions. For more information about how Microsoft IT designed and deployed
Office Live Communications Server 2005 with SP1, refer to
http://www.microsoft.com/downloads/details.aspx?familyid=61e90c28-3840-4c89-9b19-2814a6a62aaf&displaylang=en.
The deployment options and design decisions that Microsoft IT made for the transition
to Office Communications Server 2007 depended on not only the existing network
infrastructure and Active Directory environment, but also on the telephony infrastructure.
Additionally, because Office Communications Server 2007 integrates with Microsoft
Exchange Server 2007 unified messaging, Microsoft IT also considered the e-mail
messaging infrastructure.
To understand the factors that contributed to the design decisions that Microsoft
IT made, it is important to understand the previous environment and infrastructure.
Figure 1 shows the previous TCP/IP network, telephony infrastructure, and messaging
environment.
.gif)
Figure 1. Network and telephony infrastructure before Office Communications Server 2007
As Figure 1 shows, Microsoft IT deployed Live Communications Server 2005 with
SP1 in the IT perimeter network and in the Redmond Active Directory site that is
part of the corporate forest. The perimeter network contained access proxies for
employee and federated partner use, and an external director, whereas the corporate
forest contained a load-balanced front-end server pool, a two-node Microsoft SQL
Server® cluster, an application server, a bridgehead server, two internal directors,
and archiving agent servers with an associated database server. Microsoft IT implemented
remote call control (RCC) by using a third-party computer-telephony integration/private
branch exchange (CTI/PBX) and public switched telephone network (PSTN) gateways
between the IP network and TDM PBX.
Other aspects of the existing environment that Microsoft IT considered for the deployment
of Office Communications Server 2007 included:
- Geographic location This consideration relates both to data
centers that house servers and to locations enabled for communications services.
Microsoft IT has three major data centers, one each in Singapore, Redmond, and Dublin, and
supports facilities in 98 countries. To accommodate the remote data centers and
provide optimal performance to users worldwide, Microsoft IT chose to geographically
disperse the topology.
- Wide area network (WAN) connectivity and bandwidth Microsoft
IT relies on Gigabit Ethernet and Synchronous Optical Network (SONET) links in the
Redmond location and WAN links of 155 megabits per second (Mbps) to the three major
data centers. For telephony connectivity, Microsoft IT uses multiple T1 lines with
channel-associated signaling (CAS)/Q.SIG signaling integration.
Opportunities with Office Communications Server 2007
As part of its mission to help product groups deliver software that is proven in
a production environment, Microsoft IT examines its IT environment from the viewpoint
of a critical customer, including performing gap analysis and considering what must
be in place to design, deploy, and operate an IT environment. For Microsoft IT,
transitioning to Office Communications Server 2007 from Live Communications
Server 2005 with SP1 provided an opportunity to take advantage of new features,
such as the Microsoft RoundTable communications and archival system and Enterprise
Voice. Together with the unified messaging capabilities in Exchange Server 2007,
Office Communications Server 2007 gives employees communications tools for
voice, IM, and conferencing in a tightly integrated way by using familiar clients
such as Microsoft Office Outlook® 2007. IT administrators and operators
also benefit from improved administrative tools and Active Directory based configuration
objects.
"Office Communications Server 2007 gives Microsoft IT the ability to offer users
more robust and flexible communication tools without drastic changes to the existing
infrastructure."
Mark Achzenick
Group Manager
Microsoft Corporation
Microsoft IT was looking forward to taking advantage of improvements in Office Communications
Server 2007, including:
- Streamline communications across IM, presence, audio/video (A/V) conferencing,
and telephony Office Communications Server 2007 brings
a more consistent user experience across a variety of communication modes, with
one interface for managing IM through conferencing and telephone calls.
- Improve conferencing options with the on-premise server Audio
conferences at Microsoft typically include three to four participants who can dial
in through a third-party provider bridge. Microsoft IT can decrease costs by using
the built-in IM, presence, and conferencing capabilities of Office Communications
Server 2007 over the IP network. In addition, new multiparty IM features enable
users to add new participants to an existing IM conversation and to escalate a conversation
to an on-premise, audio/video Web conference.
- Increase return on investment (ROI) for conferencing Microsoft
used the internal meeting services of Office Communications Server 2007 and
replaced the existing on-premise online meeting services, which resulted in a higher
ROI.
- Transition to Microsoft RoundTable RoundTable is an all-in-one
video-conferencing phone device that can capture and broadcast synchronized audio
and video of a 360-degree panoramic view of everyone in a meeting. This capability
is important for Microsoft because it breaks down communication barriers caused
by location and provides a richer meeting communication experience to participants.
- Eliminate single points of failure Microsoft IT recognized
the increasing importance of Office Communications Server 2007 as part of the
overall unified communications infrastructure, and wanted to ensure high availability
by eliminating single points of failure. The architecture of Office Communications
Server 2007 made this possible at multiple levels, such as many data centers
housing servers, multiple servers for each server role, redundant telephony links,
hardware and software load balancing, and multi-node SQL Server cluster integration.
- Increase user mobility Office Communications Server 2007
enables specified Microsoft users to make VoIP calls from anywhere in the world
with a sufficient Internet network infrastructure to support audio and video calls.
Office Communications Server 2007 supports several clients for mobile users,
including Microsoft Office Communicator Mobile, Communicator Web Access, and Microsoft
Office Communicator 2007. These clients provide the rich IM and presence functionality
that mobile users need for flexible communication on the road and at home.
- Enhance security Office Communications Server 2007
not only provides enhanced security by design because it was developed in compliance
with the Trustworthy Computing Security Development Lifecycle (SDL), but also enhances
security in the following areas:
- Network communication Most network communications in Office
Communications Server 2007 are encrypted by default. All servers use certificates
and use Kerberos authentication, Transport Layer Security (TLS), Secure Real-Time
Transport Protocol (SRTP), and other industry-standard encryption techniques to
help protect Office Communications Server 2007 data on the network. In addition,
role-based setup makes it possible to deploy Office Communications Server so that
only the services, and the permissions related to those services, are installed
as appropriate on each server role.
- Server authentication Office Communications Server 2007
relies on certificates for server authentication and to establish a chain of trust
between clients and servers and among the different server roles. The Windows Server
2003 public key infrastructure (PKI) provides the infrastructure for establishing
and validating this chain of trust.
- Improve logging and compliance Microsoft IT can use the
included call detail record (CDR) functionality to record when users log on and
log off, as well as to record details about a meeting and specific IM conversations.
CDRs enable Microsoft IT to report on call usage and thereby learn more about how
the system is being used. Even though Office Communications Server 2007 offers
the capability to record the details of all conversation, text, and audio and data
collaboration, Microsoft has implemented a subset of these details in accordance
with directions from the Microsoft legal team. Microsoft IT has deployed the Office
Communicator 2007 client in a way that enables the user to choose to either
store or not store conversations based on corporate legal guidelines.
Design Challenges and Considerations
In considering the design necessary to take advantage of the improvement opportunities,
Microsoft IT began with broad considerations that shaped the overall design such
as topology, and moved on to more focused considerations, such as specific security
settings. Microsoft IT also adopted several overall goals to guide the group in
designing the infrastructure. These goals provided a framework in which to make
decisions. For example, Microsoft IT set the goal early on to increase the service
reliability and availability by eliminating single points of failure In order to
eliminate single points of failure. Accordingly, Microsoft IT designed a topology
with multiple locations, fault-tolerant server instances, and load balancing at
the front-end and back-end servers.
"The concept of server roles and server pools as logical building blocks in our designs
really simplified planning considerations for scalability, sizing, and management.
For example, we were able to create uniform, load-balanced server pools and Edge
server designs, and use them across all data centers in the internal and perimeter
networks. Perhaps best of all, Office Communications Server 2007 supports these
logical design building blocks out of the box through server roles."
Indranil Banerjee
Senior Program Manager
Microsoft Corporation
Specifically, Microsoft IT considered the following aspects when designing an infrastructure
to support Office Communications Server 2007:
- Consolidated and expanded server pool topologies The Office
Communications Server 2007 architecture includes the concept of server pools
that reside in the internal network and consist of front-end, Web component, and
conferencing servers with a SQL back end. Microsoft IT, as part of its product validation
directive, had to deploy both an expanded server pool with multiple physical servers
hosting A/V and Web components, and a consolidated server pool where each server
hosted all conferencing components.
Note: Although Microsoft IT deployed an expanded server pool in the Redmond
data center with consolidated pools in the Singapore and Dublin data centers at
release to manufacturing (RTM), the group later transitioned the Redmond location
to use two consolidated server pools for better supportability.
- Distributed or centralized management Microsoft IT monitors
and operates the IT infrastructure centrally from the Redmond location, and it uses
regional IT teams to handle specific operational and deployment tasks. Accordingly,
Microsoft IT did not change to decentralized management with Office Communications
Server 2007; it continued to manage the entire IT infrastructure centrally.
- Coexistence and migration phases As part of its pre-release
verification tasks, Microsoft IT supported multiple pre-release builds of Office
Communications Server 2007, as well as Live Communications Server 2005
operating simultaneously.
- Site selection When considering the sites in the topology
to house Office Communications Server 2007, Microsoft IT initially kept two
aspects in mind. First, Microsoft IT considered the user needs. Locations with the
most users received higher priority. Second, Microsoft IT considered the existing
infrastructure. It is more straightforward and faster to deploy Office Communications
Server 2007 in sites that have existing VoIP gateways; therefore, those sites
received higher priority.
- Fault tolerance and eliminating single points of failure Microsoft
IT wanted to eliminate all single points of failure for the deployment of Office
Communications Server 2007, from the broadest applicable scope to the narrowest.
For example, Microsoft IT eliminated single points of failure at the data-center
level by deploying servers in multiple sites: at the connectivity level by using
redundant WAN, local area network (LAN), and PSTN links; at the server farm level
by using multiple load-balanced servers and multi-node clusters; and at the server
and storage level by using design elements such as redundant power supplies and
RAID arrays.
- Load balancing Microsoft IT considered many types of server
load. For example, Microsoft IT analyzed the typical user traffic at each location
to ensure that PSTN and WAN connections could support the conferencing load. Microsoft
IT also considered server load for front-end and back-end servers to distribute
the load. By using hardware load balancing for server pools and round-robin load
balancing for Edge servers, Microsoft developed a load-balanced design.
- Flexibility and scalability With server roles, Microsoft
IT can readily scale up both individual servers by changing server designs and site
capacity by adding new servers. For example, to handle increased federated partner
traffic, Microsoft IT can deploy additional Edge servers in the perimeter network.
Using multiple sites also increases disaster recovery flexibility because of additional
site failover choices.
- Server sizing Microsoft IT combined sizing recommendations
from the product group with previous experience in sizing servers and preliminary
testing to arrive at the initial server designs for each role. In the end, Microsoft
IT arrived at design specifications close to what the product group recommended.
(The "Server Sizing" section later in this paper provides more information
about server sizing, including the hardware specifications for each role).
- Remote access Office Communications Server 2007 introduced
the A/V Edge server for media relay, which enables remote users to access services
from outside the corporate network without connecting through a virtual private
network (VPN). Yet, for Microsoft IT to make use of this server role, it must ensure
that security is not compromised. Microsoft IT cleared the server role and specific
configurations with the corporate security group, which audited the server for security
vulnerabilities before deployment.
- Security Microsoft IT considered security at many parts
of the design. At the topology level, Internet-facing front-end Edge servers reside
in a perimeter network, which helps to protect internal resources from being compromised.
Office Communications Server 2007 comes with encryption capabilities, such
as Mutual Transport Layer Security (MTLS), for traffic. Microsoft IT considered
the appropriate encryption method for functions such as IM, Web conferences, audio,
and external access. Microsoft IT configured both internal certification authorities
(CAs) and external or public CAs for data encryption.
- Disaster recovery Even though Microsoft IT eliminated single
points of failure, it still considered disaster recovery plans. In the worst-case
scenario of data-center failure, Microsoft IT can recover by using other data centers
to handle the load. Additionally, Microsoft IT created comprehensive documentation
about the detailed design decisions in addition to deployment checklists, which
enable engineers to repeat configuration and setup steps in case of catastrophic
failure.
- Call authorization and routing As part of its configuration
considerations; Microsoft IT examined the dependencies of providing service features
to users. Specifically, Microsoft IT noted that to use PC-to-phone functionality,
users must be enabled for unified communications.
- Application compatibility Microsoft IT needed to ensure
that any third-party application running on Live Communications Server 2005
would be compatible with Office Communications Server 2007. For example, Microsoft
IT needed to ensure that Remote Call Control middleware running on Live Communications
Server 2005 with SP1 was compatible with Office Communications Server 2007
to continue to deliver those services to the user base in Office Communications
Server 2007.
Environment Topology
Before even considering the specific number of servers, their placement, or their
sizing, Microsoft IT encountered design considerations that involved meeting its
commitment to the product group of verifying multiple topology options of the product.
There are two areas where Microsoft IT made topology decisions: in the perimeter
network and with the server pool in the internal network. The perimeter network
houses A/V, Web Conferencing, and Access Edge servers, whereas the internal network
houses the front-end and back-end servers for conferencing, IM, presence, storage,
and archiving.
In pursuing the goal to eliminate single points of failure and provide high availability
and scalability, Microsoft IT decided to deploy two sets of load-balanced servers,
one set for the Access and Web Conferencing roles, and the other for the A/V Edge
role. Figure 2 illustrates the Edge role server topologies. In all the topologies
for the perimeter network, servers running Microsoft Internet Security and Acceleration
(ISA) Server handle the HTTP and HTTPS traffic between external participants and
the internal server pools.
.gif)
Figure 2. Edge server role consolidated and expanded server pool topologies
In the perimeter network, it is possible to deploy one or multiple roles on each
physical server; the specific mix depends on the site topology and desired scalability.
For example, although Microsoft IT could have dedicated one or more physical servers
for each server role, this was not cost-effective because in the Microsoft IT environment,
the Web Conferencing and Access Edge roles typically experience less load than the
A/V Edge role that relays media. Consequently, Microsoft decided to deploy the Web
Conferencing and Access Edge roles together and use dedicated servers for the A/V
Edge role.
The roles in Office Communications Server 2007 enable the deployment of a consolidated
and expanded topology for the server pools. A consolidated topology helps simplify
setup and management, while still providing high availability and failure recovery
by running the Focus, Focus Factory, Conferencing Server Factory, and conferencing
server features on all front-end servers. In an expanded configuration, the A/V
Conferencing Server and Web Conferencing server roles are distributed and run on
separate servers. This configuration improves scalability by enabling Microsoft
IT to add servers of a specific type when necessary. An expanded server pool topology
enables Microsoft IT to scale conferencing servers independently of the presence,
signaling, and conference control elements that run on front-end servers. Figure
3 illustrates consolidated and expanded server pool topologies.
.gif)
Figure 3. Consolidated and expanded server pool topologies at Microsoft
Microsoft IT decided to deploy the expanded server pool topology in the Redmond
data center that serves approximately 80,000 users in North and South America. The
team also decided to initially deploy a consolidated topology for the two other
data centers in Singapore and Dublin, with the goal of transitioning all sites to
a consolidated server pool topology after completing product verification. You can
find more details about the topology and Microsoft IT's reasons for making specific
deployment decisions in the section titled "Global Topology".
Distributed or Centralized Management
Before deploying Office Communications Server 2007, Microsoft IT followed a
centralized model for design, deployment, and operations. This entailed using dedicated
teams that handled a specific product, and shared service teams that performed product-independent
processes, such as front-line server monitoring. For Office Communications Server 2007
specifically, Microsoft IT saw no business reason to use a distributed management
model where each site managed its own design and deployment processes. Just the
opposite, the Office Communications Server team in Microsoft IT handles all the
design, deployment, and operations aspects. When necessary, the team uses third-party
providers and local, on-site IT teams for some unified communications related deployment
aspects, such as VoIP gateway installation and telephony integration.
Migration Phases
Deploying all new services at Microsoft occurs in phases; this is a Microsoft IT
best practice because it enables fine-tuning of the design, minimizes user impact
if changes must be made, and mitigates the risk of deploying new services. Microsoft
IT identified two levels at which it could phase in Office Communications Server
deployment: the data-center level and the client level. Microsoft IT planned to
deploy the hardware in the three data centers simultaneously in order to provide
users with the new services worldwide. Each data center handles traffic for specific
geographic locations. For example, the Singapore data center handles traffic from
Asia and Australia, whereas the Dublin data center handles traffic from Europe and
Africa.
After deploying the hardware and configuring Office Communications Server 2007,
Microsoft IT began a client-level phased-in migration. Microsoft IT accomplished
this by first identifying multiple locations homed to each data center, and then
identifying groups of users in each location to migrate. To ensure reasonable adoption
rates, Microsoft IT targeted one department or building at each location at a time.
Users from specific departments or buildings formed the initial user pool. At the
client level with the user pool, Microsoft IT pursued the following methodology:
- Phase I Migrate the users in each department or building
who are part of Microsoft IT to minimize risk and impact.
- Phase II Expand the pilot to additional users picked randomly
from the same locations.
- Phase III Expand the pilot to more locations homed at each
data center.
This approach reduced the risk with regard to the installation of a new client and
to how the client would work in a production environment. Most importantly, the
gradual ramp-up approach provided a gating factor in terms of support. Microsoft
IT was able to identify issues with smaller batches of users, fix those issues,
and then move forward with the deployment to larger batches of users.
Site Selection
As already mentioned, Microsoft IT considered the number of users and existing infrastructure
when choosing the initial sites to be enabled for unified communications with Office
Communications Server 2007. Yet, these were not the only considerations. For
example, Microsoft IT also made sure that the site had adequate bandwidth to support
the additional traffic, both peer to peer, and client to mediation server. On the
telephony side, the PBX needed to have Q.SIG T1/E1 ports for connecting to the VoIP
gateway. Additionally, Microsoft IT made sure that all potential locations were
cleared of regulatory and homologation hurdles for gateway and client devices.
Note: In some countries, use of VoIP is illegal. Enterprises considering
a deployment to a foreign country should ensure that the country allows VoIP. They
should also ensure that the local governments in those countries have qualified
and approved devices that run Office Communications Server 2007.
Load Balancing
Microsoft IT wanted to ensure that users experienced a high quality of service with
Office Communications Server 2007. Although sizing servers properly, distributing
the topology to multiple data centers, and ensuring adequate bandwidth provided
a sound foundation for a high quality of service, using load balancing at multiple
points gave Microsoft IT yet another level of control for optimization. Microsoft
IT decided to use both hardware load balancers and Domain Name System (DNS) round
robin in its design. Specifically, Microsoft IT uses the following strategies with
these server roles:
- Server pool Microsoft IT uses a third-party hardware load
balancer for the front-end servers in the server pool. This is the case with both
the expanded and consolidated configurations. Using two server pools in each data
center in the consolidated configuration provides load balancing in addition to
redundancy.
- A/V Edge servers Microsoft IT uses DNS round robin to balance
the media traffic that the A/V Edge servers handle. Microsoft IT chose to use round
robin (an unsupported configuration) initially instead of a hardware load balancer
for expediency in order to deploy A/V Edge servers without the need for the additional
training and support requirements associated with a hardware load balancer. In the
future, Microsoft IT plans to implement a hardware load balancer for A/V Edge servers.
- Director servers Microsoft IT uses a third-party hardware
load balancer for servers that host the Director role.
- Access and Web Conferencing Edge servers The servers that
host the Access Edge and Web Conferencing Edge roles are load-balanced with a third-party
hardware load balancer.
Server Sizing
Office Communications Server 2007 introduced several new server roles, such
as the A/V Conferencing server, Web Conferencing server, and IM Conferencing server,
to support the new on-premise Web conferencing and Enterprise Voice functionality
that Microsoft IT could include in its infrastructure. Specifically, the following
roles are available:
- Access Edge server Formerly known as the Access Proxy, this
server handles all Session Initiation Protocol (SIP) traffic that crosses the corporate
firewall. The Access Edge server is required for all external user scenarios, including
conferencing, remote user access, federation, and public IM connectivity.
- Web Conferencing Edge server This server proxies Persistent
Shared Object Model (PSOM, a Microsoft Office Live Meeting protocol) traffic between
the Web Conferencing server and external clients. The Web Conferencing Edge server
must authorize external conference traffic before it is forwarded to the Web Conferencing
server. The Web Conferencing Edge server requires external clients to use TLS connections
and obtain a conference session key.
- A/V Edge server This server provides a single trusted connection
point through which audio and video traffic enters and exits a network. With an
A/V Edge server, users can add audio and video data to meetings with external participants,
and share audio and video directly with an external user (point-to-point). The A/V
Edge server also handles audio for Enterprise Voice.
- Mediation server This server acts as a bridge between the
Enterprise Voice components and the media gateway, IP-PSTN gateway (a partner product).
A Mediation server will route a PSTN call to the appropriate Enterprise Voice client
and vice versa.
- Front-end server This server houses instant messaging, presence,
telephony, conferencing servers, and all essential user services operations.
- Conferencing server Supporting multiparty conferences requires
a new server role known as a conferencing server (also known as a multipoint control
unit or MCU). A conferencing server is a pluggable component that manages one or
more media types. Office Communications Server 2007 includes four conferencing
servers and the extensible architecture for adding more:
- IM Conferencing server Enables group IM by relaying IM traffic
between all participants. When a third participant is added to a peer-to-peer IM
conversation, the initiating client invites the IM Conferencing server to the conversation.
From that point, all messages between the participants are routed through the IM
Conferencing server. The IM Conferencing server is an integral part of the front-end
server and cannot be installed on a separate computer.
- Telephony Conferencing server Responsible for audio conferencing
provider (ACP) integration. Supports both dial-out and dial-in, in addition to standard
third-party call control features such as mute and eject. The Telephony Conferencing
server does not support mixing VoIP and PSTN in the same call. Connecting dial-out
to PSTN endpoints requires a Mediation server, as described later in this paper.
The Telephony Conferencing server is an integral part of the front-end server and
cannot be installed on a separate computer.
- Web Conferencing server Manages conference data collaboration,
including native support for Microsoft Office PowerPoint® presentations, Microsoft
Office document sharing, white boarding, application sharing, polling, Q&A,
meeting compliance logging, annotations, handouts, and various multimedia formats.
The Web Conferencing server uses PSOM for uploading slides to a meeting. The Web
Conferencing server can reside either on the front-end server (Standard Edition
and Enterprise Edition Consolidated Pool) or on a separate physical computer (Enterprise
Edition Expanded Pool).
- A/V Conferencing server Provides multiparty IP audio and
video mixing and relaying, including RoundTable, by using industry-standard Real-Time
Transport Protocol (RTP) and Real-Time Transport Control Protocol (RTCP). The A/V
Conferencing server can reside either on the front-end server (Standard Edition
and Enterprise Edition Consolidated Pool) or on a separate physical computer (Enterprise
Edition Expanded Pool).
- Web Components server This server enables users to access
meeting presentations and other content from Web conferences; download Address Book
server files to provide Office Communicator with global address list information;
expand membership in distribution groups; and obtain other data that the Web Conferencing
server uses.
- Director The director server is a server role that can be
deployed in standard or enterprise deployments with remote or federated users who
access internal servers. The Director role offloads authentication tasks from front-end
servers by running only authentication services. In this way, front-end servers
can focus resources on Web conferencing, A/V conferencing, and IIS services. Microsoft
IT decided to use one Director server running Microsoft Office Communications Server 2007
Standard Edition in the Redmond data center.
- Microsoft Office Communicator Web Access Office Communicator
Web Access provides a server scale web client for Office Communications Server that
targets the following three platforms: security-hardened computers running the Windows®
operating system with no permissions to install Office Communicator 2007; computers
not running a Windows operating system that cannot install Office Communicator 2007;
and mobile devices that have Web browsing capability and are compatible with Asynchronous
JavaScript and XML (Ajax) technology or custom clients.
Both Microsoft IT and the Office Communications Server 2007 product group performed
repeated load tests on all the server roles by using early code versions of the
software to determine how many users could be supported and the hardware requirements
for each server role. They used these numbers to design the initial architecture.
To ensure that Microsoft IT was able to meet the high capacity demands associated
with supporting a growing number of users, it used dedicated servers, as shown in
Table 1.
Table 1. Server Specifications for Server Roles in Office Communications Server
|
Role
|
Consolidated
|
Expanded
|
|
Front End
|
4 2.33-gigahertz (GHz) processor, plus 8 gigabytes (GB) of RAM
|
2 2.33-GHz processor, plus 4 GB of RAM
|
|
File Server
|
4 2.33-GHz processor, plus 8 GB of RAM
|
4 2.33-GHz processor, plus 4 GB of RAM
|
|
Communicator Web Access
|
Not applicableחsingle Communicator Web Access server in Redmond
|
2 2.33-GHz processor, plus 4 GB of RAM
|
|
A/V Conferencing
|
Not applicableחconsolidated
|
4 2.33-GHz processor, plus 8 GB of RAM
|
|
Data (IM, Web, Telephony) Conferencing
|
Not applicableחconsolidated
|
4 2.33-GHz processor, plus 8 GB of RAM
|
|
Access Edge
|
2 2.33-GHz processor, plus 8 GB of RAM
|
2 2.33-GHz processor, plus 8 GB of RAM
|
|
A/V Edge
|
2 2.33-GHz processor, plus 8 GB of RAM
|
2 2.33-GHz processor, plus 8 GB of RAM
|
|
SQL
|
4 2.33-GHz processor, plus 16 of GB RAM
|
4 2.33-GHz processor, plus 16 GB of RAM
|
|
Archiving
|
4 2.33-GHz processor, plus 8 GB of RAM
|
4 2.33-GHz processor, plus 8 GB of RAM
|
|
Director
|
2 2.33-GHz processor, plus 4 GB of RAM
|
2 2.33-GHz processor, plus 4 GB of RAM
|
|
Mediation
|
4 2.33-GHz processor, plus 8 GB of RAM
|
4 2.33-GHz processor, plus 8 GB of RAM
|
|
Web Components
|
Not applicableחconsolidated
|
2 2.33-GHz processor, plus 8 GB of RAM
|
After arriving at the initial processor and memory configuration for each server
type, Microsoft IT needed to determine the number of servers required at each data
center to support the users. To determine the number of servers at each location,
Microsoft IT had to know the maximum number of concurrent users each server could
support with no loss in service quality, and the total number of users homed to
each data center.
Although determining the number of users homed to a data center was straightforward
(Microsoft IT counted the total users at each site homed to a data center and calculated
the sum, which came to 24,118 for Dublin, 24,649 for Singapore, and 86,587 for Redmond),
determining the maximum concurrent users proved more of a challenge because that
depends on the usage patterns. For example, one usage pattern may be that federated
partners rarely hold videoconferences, and all internal users use the Office Communicator 2007
client. Another pattern may have users using primarily IM and no audio or video.
Therefore, Microsoft IT made assumptions about typical usage based on statistics
gathered from operating Live Communications Server 2005 with SP1. Among other
things, Microsoft IT considered these usage pattern factors:
- Client choice As mentioned later, Office Communications
Server supports many devices, including Office Communicator 2007, Communicator Web
Access, and Office Communicator Phone Edition (VoIP) endpoints. Microsoft IT used
these three clients because they are the most commonly used types. Microsoft IT
assumed that at any time at any location, 100 percent of the users were connected
through Office Communicator 2007, 5 percent were connected through a VoIP endpoint
device, 4 percent were connected through Office Communicator 2005, and 1 percent
were connected through Communicator Web Access.
- Active users Microsoft IT estimated that approximately 65
percent of users homed to a specific data center were actively using Office Communications
Server 2007. To make room for growth, Microsoft IT added 10 percent to the
active user estimates, arriving at 18,089 for the Dublin data center, 18,487 for
the Singapore data center, and 65,940 for the Redmond data center.
- Server role connections The usage patterns differ depending
on the server role to which a client is connected. For example, whereas an Office
Communicator 2007 client may be connected to an Access Proxy Edge role server
5 percent of the time, an Office Communicator 2005 client may be connected
to an Access Proxy Edge role server 100 percent of the time. Table 2 shows the usage
model Microsoft IT used for the consolidated and expanded topologies per role and
client type.
Table 2. Usage Model per Role and Client Type
|
Component
|
Office Communicator 2007
|
VoIP endpoint
|
Office Communicator 2005
|
Communicator Web Access
|
|
Expanded topology
|
|
Active users
|
100%
|
5%
|
4%
|
1%
|
|
Front-end
|
100%
|
100%
|
100%
|
100%
|
|
Back-end
|
100%
|
100%
|
100%
|
100%
|
|
Communicator Web Access
|
0%
|
0%
|
0%
|
100%
|
|
Archiving Database
|
100%
|
100%
|
100%
|
100%
|
|
File Server
|
100%
|
100%
|
100%
|
100%
|
|
Quality of Experience (QoE)
|
0%
|
0%
|
0%
|
0%
|
|
Web Conferencing
|
0%
|
0%
|
0%
|
0%
|
|
A/V MCU
|
4%
|
4%
|
100%
|
0%
|
|
Data MCU
|
0%
|
0%
|
100%
|
0%
|
|
A/V Edge
|
1%
|
4%
|
100%
|
0%
|
|
Web Edge
|
0%
|
0%
|
100%
|
0%
|
|
Access Proxy Edge
|
5%
|
4%
|
100%
|
0%
|
|
Edge Data Relay
|
5%
|
4%
|
100%
|
0%
|
|
Consolidated topology
|
|
Active Users
|
100%
|
5%
|
4%
|
1%
|
|
Front-end + A/V + Data + Web Conferencing
|
100%
|
100%
|
100%
|
100%
|
|
BE
|
100%
|
100%
|
100%
|
100%
|
|
Communicator Web Access
|
0%
|
0%
|
0%
|
100%
|
|
AA
|
100%
|
100%
|
100%
|
100%
|
|
Archiving Database
|
100%
|
100%
|
100%
|
100%
|
|
File Server
|
100%
|
100%
|
100%
|
100%
|
|
QoE
|
0%
|
0%
|
0%
|
0%
|
|
A/V Edge
|
1%
|
4%
|
100%
|
0%
|
|
Web Edge
|
0%
|
0%
|
100%
|
0%
|
|
Access Edge
|
5%
|
4%
|
100%
|
0%
|
After Microsoft IT created a usage model and knew the total number of active users
homed to each database, it was able to calculate the number of servers required.
To do this, Microsoft IT first obtained recommendations from the product group about
the number of simultaneous connections each server type could accept without performance
degradation, and then calculated the typical number of connections based on the
usage model. This information provided the required quantity of servers for each
server type in both expanded and consolidated topologies.
For example, for the Redmond data center, Microsoft IT calculated the number of
simultaneous connections for servers by first calculating the typical number of
clients connected for each client type (64,940*100%=64,940; 65,000*5%=3,247; 65,000*4%=2,598;
65,000*1%=649), and then calculating the typical number of clients connected for
the specific server role (for front-end servers, because the usage is 100 percent
for all four client types, this would be 64,940+3247+2,598+649=70,785). Because
front-end servers in the expanded topology can handle up to 10,000 concurrent sessions,
it takes 71,500/10,000=7.15 servers to handle the load. Correspondingly, Microsoft
IT planned to deploy eight front-end servers in the Redmond data center. Table 3
shows the maximum number of concurrent sessions per server role and the number of
sessions per data center.
Table 3. Server Specification for All Server Roles in Office Communications Server
|
Topology or category
|
Role
|
Maximum concurrent sessions per server
|
Dublin sessions
|
Singapore sessions
|
Redmond sessions
|
|
Expanded
|
Front End server
|
10,000
|
Not applicable
|
Not applicable
|
70,785
|
|
Back End SQL server
|
50,000
|
Not applicable
|
Not applicable
|
70,785
|
|
Web Component server
|
25,000
|
Not applicable
|
Not applicable
|
0
|
|
A/V Conferencing server
|
525
|
Not applicable
|
Not applicable
|
5,325
|
|
Data (IM, Web, Telephony) Conferencing server
|
625
|
Not applicable
|
Not applicable
|
2,598
|
|
Consolidated
|
Back End SQL server
|
25,000
|
19,897
|
20,335
|
Not applicable
|
|
Front End, Web Component, A/V Conferencing, and Data Conferencing server
|
5,000
|
19,897
|
20,335
|
Not applicable
|
|
Standard Edition
|
Front End, Web Component, A/V Conferencing, and Data Conferencing with SQL Server
Desktop Engine or Microsoft SQL Server 2005 Express Edition server
|
5,000
|
Not applicable
|
Not applicable
|
Not applicable
|
|
Edge
|
Stand-Alone Access Edge
|
10,000
|
1,664
|
1,701
|
5,975
|
|
Stand-Alone Web Conferencing Edge
|
3,000
|
724
|
739
|
2,598
|
|
Stand-Alone AV Edge
|
1,500
|
941
|
961
|
3,377
|
|
Collocated Access and Web Edge
|
6,500
|
Not applicable
|
Not applicable
|
5,975
|
|
Collocated Access, Web, and A/V Edge
|
5,250
|
Not applicable
|
Not applicable
|
Not applicable
|
|
Other
|
Archiving Agent
|
100,000
|
19,897
|
20,335
|
70,785
|
|
Archiving Database
|
100,000
|
19,897
|
20,335
|
70,785
|
|
File Server
|
100,000
|
19,897
|
20,335
|
70,785
|
|
Communicator Web Access
|
Not applicable
|
181
|
Not applicable
|
Not applicable
|
|
Director
|
Not applicable
|
Not applicable
|
Not applicable
|
Not applicable
|
|
Mediation
|
Not applicable
|
Not applicable
|
Not applicable
|
Not applicable
|
With these calculations of the actual number of servers necessary for each data
center, Microsoft IT made adjustments as necessary to achieve load balancing, redundancy,
high availability, and other goals. For example, although theoretically two Edge
servers in the perimeter network could handle the load for the Singapore data center,
Microsoft IT deployed Edge servers load balanced in pairs to provide high availability.
Microsoft IT's server design is only one possibility with Office Communications
Server 2007 and does not necessarily reflect the capacity limits supported by the
product. For more information about design planning, refer to the Office Communications
Server 2007 Planning Guide at
http://www.microsoft.com/downloads/details.aspx?familyid=723347C6-FA1F-44D8-A7FA-8974C3B596F4&displaylang=en.
Remote Access
The three Edge server roles and the reverse proxy server provide remote capabilities
for federated partners and other remote users who participate in audio and video
conferences, as explained earlier. These servers communicate by using a variety
of protocols, such as RTP, MTLS, SIP, and Hypertext Transfer Protocol over Secure
Sockets Layer (HTTPS), through the associated ports. Accordingly, Microsoft IT submitted
the required firewall settings to the security team, which approved the open ports.
Figure 4 illustrates the ports used for communication with each Edge server role.
.gif)
Figure 4. Firewall open ports in perimeter network
Microsoft IT decided to use the Redmond location as the primary access point for
all users and consequently deployed the Access Edge server role along with the Web
Conferencing and A/V Edge roles in the Redmond data center. To eliminate the single
point of failure for remote access in Redmond, Microsoft IT decided to deploy the
Access Edge role in the Dublin location as well, but only as a failover in case
the Access Edge servers in Redmond are unavailable. The Communicator 2007 client
natively supports retrying all servers returned by DNS if a primary server is unavailable.
By specifying the Access Edge servers in Redmond as primary and the Access Edge
server in Dublin as secondary, Microsoft IT makes use of the built-in bootstrap
functionality in the Communicator 2007 client for failover.
In the Singapore location, Microsoft IT deployed the Web Conferencing and A/V Edge
roles. Because the WAN backbone supports transfer speeds of 155 Mbps and higher,
Microsoft IT did not encounter latency issues with communication.
Security
Even with having the built-in authentication and encryption capabilities in Office
Communications Server 2007, Microsoft IT needed to consider and design the
proper strategy for having a security-enhanced environment at all levels. As with
other design considerations, Microsoft considered the security aspects systematically
by tracking the flow of data and authentication protocols inside and outside the
corporate network. Microsoft IT considered the following security aspects in its
design:
- Internet boundary Microsoft IT explicitly defines the Edge
servers as trusted communication partners with internal servers by using security
certificates, MTLS, and defined server lists in Active Directory. The Director role
helps to increase security by authenticating external enterprise users and routing
these users to their home pools. Because Access Edge servers do not communicate
with Active Directory, using them in combination with Directors helps defend against
distributed Internet attacks, where attackers are posing as remote users. Each of
the three regional data centers provides its own perimeter network for Edge servers.
However, the Redmond data center houses the primary Access Edge servers used for
remote access, whereas the Dublin data center houses the failover Access Edge servers.
- On-premise conferences Using on-premise Web conferences
was a key goal for Microsoft IT, yet meeting the goal included security considerations.
Office Communications Server 2007 provides built-in safeguards to help reduce
risks, such as limited access and control based on participant roles, meeting types
with access controls for participant types, conference scheduling restricted to
users enabled in Active Directory, and unique passwords for unauthenticated users
to join meetings.
- Enterprise Voice Microsoft IT decided to deploy Mediation
servers in separate virtual LANs (VLANs) to help control network traffic and provide
better segmentation and QoS capabilities. Microsoft IT also takes advantage of the
built-in security by configuring Mediation servers to use MTLS and SRTP for all
communications.
- General network infrastructure and server hardening Microsoft
IT hardens all servers as part of general security practices. For example, Microsoft
IT hardens the operating system by disabling unused services and enabling communication
only on the necessary ports. Microsoft IT uses Internet Protocol security (IPsec)
in the network as part of general security measures. To support the required quality
of service for communications, Microsoft IT configures Office Communications Server 2007
to use IPsec but not run in required mode.
- Office Communicator client Microsoft IT uses Group Policy
settings to enforce security for clients. For example, Microsoft IT enforces media
encryption and encrypted signaling by using TLS.
Active Directory handles user authentication, and Microsoft IT uses both NTLM and
Kerberos as authentication protocols. Although Microsoft IT prefers the use of Kerberos,
it must use NTLM to support earlier platforms.
All traffic passing through the pool of front-end servers has a Secure Sockets Layer
(SSL) connection and a high level of encryption. This includes IM and presence sessions,
in addition to Web conferencing traffic. Office Communications Server 2007
requires certificates to be installed on each server that uses MTLS, which is used
to handle security for server-to-server communications. This requirement ensures
that each server will validate the other server's certificate. For communications
between the server and a client, Microsoft IT uses the TLS protocol.
Supported Clients
Microsoft IT considered the client availability with Office Communications Server 2007
and decided to initially support the following clients and devices:
- Office Communicator 2007 An enhanced version of the
client that came with Live Communications Server 2005, Office Communicator 2007
provides IM, group IM, and other enhanced capabilities.
- Office Communicator Web Access This enables users to access
IM and presence information over the Internet via HTTPS.
- Office Live Meeting This newly available client gives internal
and external users access to view and hear the A/V that is associated with a meeting.
Users can also view information that is made available through data collaboration,
in addition to the presenters' shared application.
- Conferencing add-in for Office Outlook By using this add-in,
users who schedule on-premise meetings will view a user interface that is similar
to the one they use when scheduling meetings via Office Outlook.
- RoundTable This newly introduced conferencing phone provides
a 360-degree view of a conference for external attendees. In conjunction with Office
Live Meeting Console, this device provides a panoramic view of the conference and
an active speaker window.
- Communicator Mobile This is the client for Windows Mobile®
5.0 software.
Office Communications Server 2007 Implementation
"Implementing Office Communications Server 2007 was challenging because it integrates
e-mail messaging, the IP network, and the traditional PSTN. Even with proper planning,
it is impossible to anticipate all eventualities. We mitigate the risk in part through
a pilot deployment to a limited user pool before rolling out services to all users."
Bharath Sankaranarayan
Group Program Manager
Microsoft Corporation
After designing the environment, Microsoft IT considered the implementation challenges
in order to systematically deploy the designed infrastructure. Microsoft IT considered
the following specific, high-level deployment challenges:
- Client version control Because Microsoft IT deployed multiple
code builds in its rollout phases, users had various client builds. Although the
latest builds supported prior client versions, Microsoft IT wanted to collect meaningful
statistics from uniform client versions. Microsoft IT dealt with this situation
by taking measures to increase user adoption of the latest client version by using
Microsoft System Center Configuration Manager 2007 and client version control.
Microsoft IT used Configuration Manager first because it was a less problematic
way to encourage client version adoption. Configuration Manager enabled Microsoft
IT to notify users, who then could voluntarily update their clients. This process
resulted in approximately 9,000 users upgrading. For the remaining users, Microsoft
IT enforced client version control by using the block list capability of Office
Communications Server 2007.
- Version coexistence Microsoft IT faced another challenge
in supporting both the hosted Office Live Meeting service and the on-premise Web
conferencing service. Because both versions were available, some users were confused
about which tool to use. The user adoption group within Microsoft IT prescribed
a message of "choose the best tool for the job" for users who wanted to
schedule conferences. For example, the user adoption group recommended that users
choose the Office Live Meeting hosted service tool for large conferences, such as
e-learning sessions. For small, collaborative meetings, the group recommended the
Web conferencing capabilities built into Office Communications Server 2007.
- SQL Server clustering Microsoft IT ran the Live Communications
Server 2005 with SP1 environment with clustered SQL servers by using a storage
area network (SAN) that was shared with other services, such as messaging and collaboration
and handled by the storage services team within Microsoft IT. The product group
recommended continuing using the shared SAN with clustered SQL servers for the expanded
topology of Office Communications Server 2007. Microsoft IT initially followed
this recommendation and deployed Office Communications Server 2007 with clustered
SQL servers in the shared SAN. After deploying clustered SQL servers with a SAN,
Microsoft IT monitored performance and determined that the shared SAN did not meet
the input/output (I/O) demands for Office Communications Server 2007. To try
to meet I/O demands, Microsoft IT made the engineering decision to dedicate spindles
in the SAN for Office Communications Server 2007. After monitoring the performance
with dedicated spindles, Microsoft IT recognized that even the dedicated spindles
did not deliver the required I/O throughput. The next logical step would have been
to use a dedicated SAN for Office Communications Server 2007. However, this
was cost-prohibitive; therefore, Microsoft IT changed the design to use direct-attached
storage (DAS). This change removed SQL clustering and provided the required throughput,
yet it did not provide the required high availability for a pool that uses an expanded
topology. Therefore, Microsoft IT decided to deploy n+1 consolidated pools in Redmond,
each with its own dedicated storage through DAS. A consequence of this decision
is the lack of automatic failover; it is manual based on certain operational criteria
of moving users from a failed pool to an active pool. At RTM, Microsoft IT did not
yet implement the n+1 consolidated pool design in the Singapore and Dublin data
centers, but Microsoft IT planned to do so.
- IPsec The Microsoft infrastructure is currently IPsec enabled;
however, using IPsec creates overhead for audio and video traffic, which generates
delays in session setup. To provide the needed quality of service, Microsoft IT
obtained an exception from the corporate IT security team for all the servers running
Office Communications Server 2007 from IPsec required mode so that servers
do not force IPsec use. Yet even after obtaining the exception, implementing the
exception process was challenging. Microsoft IT implemented a new Group Policy object
(GPO) policy to maintain the exception on the required computers running Office
Communications Server 2007. Any new servers being added to the environment
had to be put in the exception list, followed by security approval.
Deployment Phases
Microsoft IT deployed multiple code builds in phases to the three data centers.
It also added users in phases to expand the test pool, adding the most users as
the product neared completion. Initially, Microsoft IT decided to deploy to the
pre-release environment and the corporate production environment. Because Microsoft
IT was not ready to deploy the product to all users, the team decided to support
Live Communications Server 2005 in parallel with Office Communications Server 2007
and support both versions.
Deployment Checklists
As part of following best practices and standard deployment and operations processes
based on industry frameworks such as Microsoft Operations Framework (MOF) and IT
Infrastructure Library (ITIL), Microsoft IT develops checklists for a base implementation.
After documenting its design decisions, Microsoft IT proceeded to create a prescribed
implementation and configuration document that included a deployment checklist.
The document guided Microsoft IT to accomplish the tasks shown in Table 4.
Table 4. Office Communications Server 2007 Deployment Checklist
|
Task
|
Details
|
|
Perform prerequisite steps for DNS, Active Directory, and SQL
|
Microsoft IT uses scripts and command-line parameters to install and import initial
configurations and prepare the environment for Office Communications Server 2007.
For example, to prepare the Active Directory infrastructure, Microsoft IT runs the
lcscmd /forest /Action:SchemaPrep, lcscmd /forest /Action:ForestPrep, and lcscmd
/forest /Action:DomainPrep commands.
|
|
Install certificates
|
Microsoft IT configures certificates for both IIS and Office Communications Server 2007.
The Office Communications Server certificate is used for MTLS session creation.
Every server that uses SIP must have this certificate configured. To install the
certificate, Microsoft IT engineers open Computer Management, right-click Office
Communications Server under Service and Applications, and then use the Certificates
menu. The IIS certificate is used for SSL session creation. The Web Components service
uses IIS, which requires this certificate to be installed. Microsoft IT configures
this on the Default Web Site in IIS.
|
|
Deploy consolidated servers
|
Microsoft IT uses a script to deploy the consolidated server for the Singapore and
Dublin data centers. The script includes the configuration parameters similar to
setupee.exe /onelinesetup /SKIPSTATECHECK /INSTALL:EE(INSTALLDIR="<directory>"),WEB(INSTALLDIR="<directory>"),AVMCU(INSTALLDIR="<directory>"),DATAMCU(INSTALLDIR="<directory>").
After deploying each consolidated server, Microsoft IT manually activates the Web
components, Web conferencing, and A/V conferencing, in addition to changing the
TLS listening port to 5065 in the Listener configuration until User Replicator is
completed.
|
|
Deploy first server in expanded configuration
|
Microsoft first installs the core SIP messaging server in the Office Communications
Server environment. This service must be installed before any other in a pool because
all other services depend on this service being operational. To install this service,
Microsoft IT runs setupee.exe /onelinesetup /SKIPSTATECHECK /INSTALL:EE(INSTALLDIR=
"<directory>").
|
|
Deploy Web component services
|
To deploy the Web Component role, Microsoft IT runs the setupee.exe /onelinesetup
/SKIPSTATECHECK /INSTALL:WEB(INSTALLDIR="<directory>") command.
|
|
Deploy Web conferencing services
|
To deploy the Web Conferencing role, Microsoft IT runs the setupee.exe /onelinesetup
/SKIPSTATECHECK /INSTALL:DATAMCU(INSTALLDIR="<directory>") command.
|
|
Deploy audio/video conferencing services
|
To deploy the A/V conferencing role, Microsoft IT runs the setupee.exe /onelinesetup
/SKIPSTATECHECK /INSTALL:AVMCU(INSTALLDIR="<directory>") command.
|
|
Deploy Edge servers
|
Microsoft IT deploys the Edge server roles on separate servers in each data center.
However, only the Redmond data center has the Access Edge role installed, with the
Dublin data center acting as failover for Access Edge servers. To install the Access
Edge role servers, Microsoft IT runs the setupee.exe /onelinesetup /SKIPSTATECHECK
/INSTALL:AP(INSTALLDIR="<directory>") command and then activates
it by using the lcscmd /server /action:Activate /Role:AP /components: command.
|
|
Deploy Director
|
Microsoft IT installs the Director role by running the setupse.exe /onelinesetup
/SKIPSTATECHECK /INSTALL:SE(INSTALLDIR="<directory>") command.
|
|
Deploy archiving
|
Microsoft IT installs archiving by running the SetupEE.exe /onelinesetup /skipstatecheck
/install:ARCH(INSTALLDIR="<directory>") command.
|
|
Deploy mediation servers
|
The mediation servers are used for unified communications telephony. They are paired
with a gateway. Microsoft IT installs the Mediation role by running the setupee.exe
/onelinesetup /SKIPSTATECHECK /INSTALL:MS(INSTALLDIR="directory>")
command.
|
These deployment details are only a start. Microsoft performs additional tasks such
as deploying and configuring load balancing. For a more comprehensive list of deployment
tasks, refer to "Overview of Enterprise Edition Deployment" at
http://technet.microsoft.com/en-us/library/bb663663.aspx.
Global Topology
By the time Microsoft IT deployed the RTM build, the environment supported 75,000
Office Communicator 2007 and Office Live Meeting users with 15,000 concurrent
IM connections, 7,500 Office Live Meeting conferences, and 2,500 active Office Live
Meeting organizers logged on per month. Microsoft IT initially deployed an expanded
topology in the Redmond data center, as shown in Figure 5. However, in the
months following the implementation of the RTM build, Microsoft IT transitioned
the Redmond data center to use a consolidated topology as well, as shown in Figure 6.
There were several reasons for transitioning the Redmond data center to use a consolidated
topology. As already mentioned, Microsoft IT determined that the shared SAN environment
did not provide the necessary I/O throughput, which resulted in a configuration
of SQL servers that used DAS without clustering. This represents a single point
of failure for each server pool and an unacceptable risk for Microsoft IT. To provide
full system redundancy, Microsoft IT decided to use multiple server pools at each
site, in addition to spreading out the user load across multiple geographically
dispersed data centers.
Another reason reasons for transitioning the Redmond data center to use a consolidated
topology is simplification. For Microsoft IT, it is more straightforward to implement,
administer, and operate a server topology that is homogenous across all data centers.
In analyzing the most common reasons for unavailability, Microsoft IT noticed that
external dependencies such as the underlying network, hardware load balancers, Active
Directory, and so on were responsible. By reducing the complexity of the environment,
Microsoft IT reduces the potential for unavailability caused by external dependencies.
.gif)
Figure 5. Microsoft IT global topology at RTM
.gif)
Figure 6. Microsoft IT global topology after RTM
Note: At RTM in August 2007, Microsoft IT did not yet fully deploy Enterprise
Voice to all users throughout the United States, Europe, and Asia. However, Microsoft
IT deployed the necessary architecture and server support for Enterprise Voice in
preparation for providing users with this functionality sometime in 2008.
Deploying New Clients and Devices
The Office Communications Server 2007 beta release included the deployment
of the new Office Communicator and Office Live Meeting clients to 15,000 of the
new users. Microsoft IT conducted an adoption campaign to inform the users of the
value and benefits of using the new clients. Due to the success of the adoption,
Microsoft IT deployed the new clients and devices to 75,000 users through a phased-in
approach, in which the next set of users received the new clients and devices only
after Microsoft IT resolved any outstanding issues with the existing user pool.
Microsoft IT initially deployed the beta release to 13,000 Microsoft Office Communicator 2005
and Office Live Meeting users. It also deployed 144 RoundTable units. In a three-month
period, users scheduled more than 1,300 Office Live Meeting conferences. Subsequent
releases would increase the number of enabled users to 48,000, with 244 RoundTable
devices and more than 4,800 Office Live Meeting conferences in another seven-week
period.
Microsoft IT deployed 25 of the 244 RoundTable devices globally to determine the
impact that these devices would have on WAN traffic. After evaluating the results,
Microsoft IT made plans to implement several thousand additional RoundTable devices
in the months following the RTM release. As already mentioned, Microsoft IT ensured
that the devices were approved for use in each respective country before deploying
them to users.
Deployment Verification
After installing and configuring Edge servers, the server pool, and additional servers
for added features such as archiving, Microsoft IT verifies the functionality and
features by using a test account and a pool of test users. By the time Microsoft
IT performs a detailed test of the environment, Microsoft IT engineers have already
configured the firewalls to permit traffic, and they have configured mediation servers
with VoIP gateways to forward data. At this point, Microsoft IT tests every location
by using the checklist shown in Table 5. IT also tests client functionality,
as shown in the checklist in Table 6 and Table 7. Microsoft IT records the result,
and if a test did not pass, provides a reason. For each location, Microsoft IT also
tests bi-directional audio capability and call setup time.
Table 5. Testing Checklist for Each Location
|
Case
|
Scenario
|
Pass/ conditionally pass/ fail
|
Reason
|
|
1
|
Make a call from inside the location to an outside number
|
|
|
|
2
|
Make a call from outside the location to an internal number
|
|
|
|
3
|
Call voice mail
|
|
|
Table 6. Testing Checklist for Office Communicator, Communicator Web Access, and
Communicator Mobile Clients
|
Case
|
Scenario
|
Pass/ conditionally pass/ fail
|
Reason
|
|
1
|
Logon
|
|
|
|
2
|
Address Book Service
|
|
|
|
|
Verify presence and IM
|
|
|
|
3a
|
Group
|
|
|
|
3b
|
Internal Dublin
|
|
|
|
3c
|
Internal Singapore
|
|
|
|
3d
|
Internal Redmond
|
|
|
|
3e
|
Internal Exchange
|
|
|
|
3f
|
Remote Dublin
|
|
|
|
3g
|
Remote Singapore
|
|
|
|
3h
|
Remote Redmond
|
|
|
|
3i
|
Remote Exchange
|
|
|
|
3j
|
Federation
|
|
|
Table 7. Checklist for Live Meeting Clients
|
Case
|
Scenario
|
Pass/ conditionally pass/ fail
|
Reason
|
|
1
|
Meet now
|
|
|
|
2
|
Internal join
|
|
|
|
3
|
Set up audio
|
|
|
|
4
|
Set up video
|
|
|
|
5
|
Remote join
|
|
|
|
6
|
Set up audio
|
|
|
|
7
|
Set up video
|
|
|
|
8
|
Upload content
|
|
|
|
9
|
Share content
|
|
|
|
10
|
Download content
|
|
|
|
11
|
Verify PSTN audio conferencing integration
|
|
|
Operations and Support
While Microsoft IT was determining the number of employees who would be affected
by each deployment, it tried to balance the interests of the Office Communications
Server 2007 product group against its own ability to handle Helpdesk calls.
The team had to overcome challenges, such as how to handle beta user support when
the Microsoft IT Helpdesk typically does not support beta products. It needed a
solution for supporting users in the best way possible while assisting the product
group with testing and with providing valuable feedback.
Tiered Support
During the deployment of Office Communications Server 2007, Microsoft IT maintained
support of Live Communications Server 2005, in addition to supporting Office
Communications Server 2007. Microsoft IT offers support in four tiers. Tier
1 answers calls placed to the Helpdesk. If necessary, a caller is routed to Tier
2, where the support staff enters a service request. If the support staff cannot
resolve the issue at the Tier 2 level, it forwards the issue to Tier 3, which handles
issues that the support materials do not cover. If Tier 3 cannot resolve the issue,
it directs the issue to Tier 4, where the problem will remain until an engineer
resolves it.
In all, the Live Communications Server 2005 and Office Communications Server 2007
installations required six engineers at the Tier 4 level, including engineers to
specifically support earlier telephony, Web telephony, and Web collaboration. Microsoft
IT needed four operations staff for Tier 3 support and an additional four support
staff for Tier 2 support. Microsoft IT expects this number to decrease after Live
Communications Server 2005 is phased out entirely.
When Microsoft IT deployed the RTM build in August 2007, the support teams
already had extensive expert knowledge for dealing with issues gathered from years
supporting Live Communications Server 2005 and from supporting beta versions
during the pilot. Microsoft IT had created enough prescribed resolution steps for
Tier 1 support members that over 80 percent of support calls were resolved at the
Tier 1 level. Table 8 shows the support tickets related to various clients
at RTM and the following two months.
Table 8. Support Ticket Percentage Distribution
|
Client product
|
August (RTM)
|
September
|
October
|
|
Communicator 2005
|
21%
|
15%
|
5%
|
|
Communicator 2007
|
65%
|
63%
|
57%
|
|
Communicator Mobile
|
1%
|
1%
|
2%
|
|
Communicator Web Access
|
<1%
|
<1%
|
<1%
|
|
Communications Server Web conferencing
|
3%
|
2%
|
5%
|
Table 9 shows the primary symptoms and factors for support tickets opened during
October. By this time, Microsoft IT experienced a significant decline in support
tickets after users had an opportunity to configure and use their clients.
Table 9. Top Primary Symptoms by Product in October, Two Months After RTM
|
Client product
|
Primary symptom
|
Percentage of Total Per Client
|
|
Communicator 2005
|
Authentication failure
|
42%
|
|
Help install/uninstall
|
32%
|
|
Presence/status issue
|
11%
|
|
IM conversation failure
|
8%
|
|
Phone integration failure
|
2%
|
|
Outlook integration issues
|
1%
|
|
Communicator 2007
|
Help install/uninstall
|
44%
|
|
Phone integration failure
|
20%
|
|
Outlook integration issues
|
8%
|
|
Presence/status issue
|
8%
|
|
IM conversation failure
|
6%
|
|
Authentication failure
|
6%
|
|
PC-to-phone audio failure
|
2%
|
|
Live Meeting add-in failure
|
2%
|
|
Application sharing failure
|
1%
|
|
Communicator Mobile
|
Authentication failure
|
39%
|
|
Help install/uninstall
|
35%
|
Administrative Support
The Microsoft IT team that supports issues related to the installation of Office
Communications Server 2007 consists of the same individuals who helped to design
and deploy the product. They are also responsible for the administration of Office
Communications Server 2007. The team uses scripting not only for deployment
but also for ongoing support. IT accomplishes scripting by using Microsoft Visual
Basic® Scripting Edition (VBScript) or Windows PowerShell command-line interface.
Microsoft IT also uses Microsoft Identity Integration Server (MIIS) to help provision
users. It used MIIS in Live Communications Server 2005 and continued to use
MIIS for the Office Communications Server 2007 deployment. MIIS can be used
to manage accounts in Active Directory.
To help support clients, an organization can use the Office Communications Server
Logger and Snooper tools. These client-side tools provide valuable information to
support staff. Office Communications Server Logger runs on the client computer and
stores information in log files that support staff can retrieve. Snooper then parses
and reads these log files.
Lessons
Learned
This section contains some of the lessons that Microsoft IT learned as it deployed
Office Communications Server 2007. It also contains suggested practices for
organizations that want to deploy Office Communications Server 2007.
- Start planning early and involve all affected groups To
plan the Office Communications Server 2007 deployment, Microsoft IT sought
input from several related teams within Microsoft IT and product groups, most notably
while implementing network perimeter changes for the data centers. For example,
some teams were responsible for DNS changes, and others were responsible for load
balancers. Microsoft IT also needed to get design approvals from the network security
team. Because multiple teams may be responsible for delivering the system requirements
in anticipation of potential delays, organizations that want to deploy Office Communications
Server 2007 should start early in their planning and involve every group that
the deployment will affect. Additionally, organizations should include the end-user
support groups early on to maximize end-user training and prepare the Helpdesk support
staff.
- Obtain executive-level support Microsoft IT obtains executive-level
support when establishing a new service at any site. For example, executives send
an e-mail message to all staff, announcing their support for Office Communications
Server 2007 and explaining the reasoning behind implementing it. Organizations
should also seek executive-level support to help drive user adoption.
- Work closely with telecom engineers when planning the Enterprise Voice implementation Organizations
should seek technical support and approval from telecom engineers during the planning
and deployment of Enterprise Voice. Microsoft IT had to work closely with gateway
and PBX partners during both the planning and deployment processes. Organizations
should also dedicate plenty of time to understanding the dial plan codes for each
new site. Certain sites, especially sites in other countries, may have different
rules regarding how to attain outside lines, or they may use a different phone number
length.
- When deploying to a new site, consider providing temporary on-site support To
help resolve issues that occur as new users are enabled, Microsoft IT found that
providing temporary on-site support was useful. This support usually involved one
support person who would physically reside in the site building.
- When deploying to clients, ensure that logging and tracing are turned on by default Microsoft
IT learned that ensuring that logging and tracing are turned on during deployment
of new clients is much easier than discovering that clients needed the services
after a problem occurs. By enabling logging and tracing on the client computer,
Helpdesk support could gather critical information that it could use to resolve
user issues.
Best Practices
The following list represents best practices for planning and implementing an Office
Communications Server 2007 deployment:
Planning
- Conduct a small pilot before rollout.
- Plan the migration of Edge servers, especially if migrating from Live Communications
Server 2005 with SP1 and running the federation service.
- Create a migration plan for all Office Communicator Mobile and Office Communicator
Web Access clients.
- Estimate bandwidth requirements for deploying RoundTable devices at planned sites.
- Use tools such as Microsoft Systems Management Server to limit user installation
mistakes and increase adoption across the organization.
- If using previous platforms, set a phase-out date for the previous version to reduce
operational support costs. The migration should have a set end date as well.
- Manage the deployment by using sufficient checklists to ensure that critical tasks
are not missed and dependencies are identified.
- Implement only the functionality that the organization needs.
Server Design and Deployment
- Document and manage all the certificates for the relevant servers, both internal
and external.
- Use A/V Edge servers for media relay instead of voice over VPN.
- Use SRTP instead of RTP to help secure audio packets.
- Use SQL clustering and highly available file shares.
- Use the service usage model in the product deployment guide to size the infrastructure.
- Client design and deployment
- For Office Live Meeting, ensure that the external link in the e-mail invitation
is active.
- Enforce Office Communicator security settings.
- Ensure that policy limits are in place for remote access users.
Infrastructure Configuration
- Ensure that the user-provisioning mechanism in MIIS is modified to the requirement
for Office Communications Server 2007. Ensure that enhanced presence is enabled
for all users.
- If IPsec is already enabled in the environment, handle all IPsec exceptions before
deployment.
- Ensure availability for remote access by using a secondary access point. As already
mentioned, for Microsoft IT, if the primary remote access entry in the United States
goes offline, the bootstrap process in the client will discover the next available
remote access point in another data center.
- Ensure that the Dynamic Host Configuration Protocol (DHCP) scopes can support the
expanded number and type of IP devices required; ensure that the total number of
devices will be supported.
- Use TLS on every device.
- Use a Gigabit Ethernet network when possible to avoid future rebuilds. Be aware
of autosensing settings and duplex settings at the infrastructure level.
- Get security approvals for the necessary topology and the required access control
lists (ACLs) and ports in the perimeter network.
User Education and Migration
- Manage the user migration in batches, with smaller batches in the beginning to reduce
the risk footprint from a support perspective.
- Provide a delay of two to three days before starting the user migration from the
time that the infrastructure is ready; this will enable the system to stabilize
before load is added.
- Run training sessions to drive adoptions with new features of the product, and primarily
with the new Office Live Meeting Console and RoundTable.
Operations
- Start measuring key operational metrics from CDR to assess system usage against
design and to drive usage adoption.
- Use Office Communications Server 2007 validation wizard deployment to validate
servers after the installation to troubleshoot connectivity issues.
- Use the post-validation tool to verify that the product was installed successfully.
Conduct phone number normalization to E.164 format before deploying the product
to production.
- Archive call detail records for auditing and metrics.
- Use version control to block previous client versions.
Conclusion
Office Communications Server 2007 presents an opportunity for Microsoft IT
to add a new and vital communications system to the existing unified communications
infrastructure. Microsoft IT previously deployed Exchange Server 2007 for e-mail
and unified messaging functionality, which helps standardize e-mail management.
Similarly, with Office Communications Server 2007, Microsoft is able to standardize
telephony management by moving away from traditional TDM-based devices and networks
to VoIP-based devices that use the TCP/IP network.
For Microsoft IT, the infrastructure design for Office Communications Server 2007
had to meet availability, reliability, and scalability requirements while eliminating
single points of failure. To achieve such a design, Microsoft IT considered many
design factors, such as server sizing, load balancing, and security. Additionally,
Microsoft IT planned to deploy Office Communications Server 2007 in phases
to minimize deployment risks.
After deploying Office Communications Server 2007, Microsoft IT continues to
evaluate the infrastructure and processes for improvement opportunities. For example,
the initial deployment used an expanded topology in the Redmond location and consolidated
topologies in the other data centers. Microsoft IT later transitioned to a consolidated
topology in the Redmond location as well for better supportability and manageability.
Microsoft IT designed the server capacity to accommodate an increase of up to 20
percent in users each year with no performance degradation. For added capacity,
it is straightforward to add additional servers as needed to the base consolidated
topologies.
Microsoft IT sees Office Communications Server 2007 as part of a strategic
investment in unified communications technology. With the base TCP/IP and telephony
infrastructure in place for many years, unified communications is a logical next
step to increase user productivity and equip employees with tools for anywhere,
anytime communication. Office Communications Server gives users the ability to communicate
on their own terms, while on the road, in the office, or at home. By demonstrating
the enterprise readiness of Microsoft unified communications products, Microsoft
IT continues to show its commitment both to Microsoft users and to real-world customers.
For
More Information
For more information about Microsoft products or services, call the Microsoft Sales
Information Center at (800) 426-9400. In Canada, call the Microsoft Canada information
Centre at (800) 563-9048. Outside the 50 United States and Canada, please contact
your local Microsoft subsidiary. To access information through the World Wide Web,
go to:
http://www.microsoft.com
http://www.microsoft.com/technet/itshowcase
http://office.microsoft.com/en-us/communicator/FX101729051033.aspx
Information in this document, including URL and other Internet Web site references,
is subject to change without notice. Unless otherwise noted, the companies, organizations,
products, domain names, e-mail addresses, logos, people, places, and events depicted
in examples herein are fictitious. No association with any real company, organization,
product, domain name, e-mail address, logo, person, place, or event is intended
or should be inferred. Complying with all applicable copyright laws is the responsibility
of the user. Without limiting the rights under copyright, no part of this document
may be reproduced, stored in or introduced into a retrieval system, or transmitted
in any form or by any means (electronic, mechanical, photocopying, recording, or
otherwise), or for any purpose, without the express written permission of Microsoft
Corporation.
Microsoft may have patents, patent applications, trademarks, copyrights, or other
intellectual property rights covering subject matter in this document. Except as
expressly provided in any written license agreement from Microsoft, the furnishing
of this document does not give you any license to these patents, trademarks, copyrights,
or other intellectual property.
© 2008 Microsoft Corporation. All rights reserved.
Microsoft, Active Directory, Outlook, PowerPoint, RoundTable, SQL Server, Visual
Basic, Windows, Windows Mobile, and Windows Server are trademarks of the Microsoft
group of companies.
All other trademarks are property of their respective owners.