Data Storage Design, Backup, and Restore for Windows SharePoint Services
| Archived content. No warranty is made as to technical accuracy. Content may contain URLs that were valid when originally published, but now link to sites or pages that no longer exist. |
Published: November 1, 2003 | Updated : September 21, 2004
By Microsoft Office Internet Platform and Operations Windows SharePoint Services Team
On This Page
Introduction
Beta Hosting Scope and Results for Data Storage
Hardware Environment
Windows SharePoint Services Storage
Hardware RAID Configuration
Server Configuration Details
SAN Storage
Backing Up and Restoring Data
Summary
Related Links
Introduction
This white paper describes the way the Internet Platform and Operations group at Microsoft configured Microsoft Windows SharePoint Services server farms to publicly host Windows SharePoint Services (Beta). This is the second of four papers that describe the Windows SharePoint Services Beta hosting experience.
The configuration and best practices outlined in this paper may be of use to anyone deploying Windows SharePoint Services (WSS) in a hosting scenario, for example, Internet service providers. This paper outlines the data storage solution including the hardware environment and RAID configuration and the database back up and restore procedures for this deployment. For more detailed descriptions and steps for backup and restore scenarios, see the Windows SharePoint Services Administrators Guide.
Beta Hosting Scope and Results for Data Storage
The Internet Platform and Operations group designed their beta deployment of Windows SharePoint Services to both test Windows SharePoint Services and provide working sites for Individual External Partner (IEP) customers.
Disk Space Requirements
One of the goals of the Windows SharePoint Services Beta hosting program was to determine how much hard disk space an average site would require. Initially, each customer site was given a quota of 30 megabytes (MB). The sever farm was set up to support 6,000 customer sites, therefore the Windows SharePoint Services content databases were designed to hold at least 180 gigabytes (GB).
Actual usage was less than anticipated. The configuration and content databases acquired only 7.5 GB. (Other space was required for the MSSQL system databases.) These results showed that initial disk space requirements were 2.5 to 3 times greater than the actual requirements. By the end of the Beta hosting program, the server farm hosted over 14,400 English, German, and Japanese sites for external world-wide customers.
Availability Requirements
The Internet Platform and Operations group had a goal of providing high availability (at least 98 percent availability for customer sites, excluding planned downtime for maintenance) and reliability and short response time. In the course of the project, based on the configuration in this series of papers, the Internet Platform and Operations group met their goals and reached 99% availability, excluding the scheduled maintenance time an excellent result for beta code.
Additional Configuration Goals
Further, the Internet Platform and Operations had the following scalability, reliability, and availability goals for the Windows SharePoint Services deployment. These areas will be addressed further in this white paper.
Validate the Windows SharePoint Services scalability design and implement a huge data store. Windows SharePoint Services supports scalability through multiple servers in server farms. To prove scalability and compatibility, the server farm must contain at least two unique content databases on two servers running Microsoft SQL Server and a storage area network (SAN) repository with more than 700 GB of raw data.
Design the backup and disaster recovery plans. Back up content and configuration information regularly and test the restoration during complete system failure situations.
Backup/Restore Design Overview
Backups are a necessary insurance for any unforeseen event or circumstances that may warrant database restore or complete disaster recovery. Developing plans and procedures for recovering from failures before they occur can also minimize damage and productivity lost. The Internet Platform and Operations group maintained regularly scheduled data and system configurations backups. Details about backing up and restoring this system are in the "Backing Up and Restoring Data" section.
Hardware Environment
The server farm used Hewlett Packard DL380 G2 and DL 580 servers. These servers were chosen because they and their parts were readily available in the laboratory, the operations personnel were familiar with the equipment, and its Compaq Insight Manager (CIM) centralized integration with the laboratorys monitoring and instrumentation tool, Microsoft Operations Manager (MOM) 2000 and MOM 2000 Service Pack 1 (SP1). Servers provided by other vendors work well in similar server farm configurations, but this paper deals with the configuration specific to the Internet Platform and Operations deployment.
Figure 1 is a diagram of the server farm and the network. For more information about configuring the servers and the hardware used in the server farm, see Microsoft Windows SharePoint Services Hosting Configuration and Experience, the first paper in this series. For more information about configuring the network and load balancing solutions, see Windows SharePoint Services Network and Load Balancing Design, the third paper in this series.
.gif "Figure 1: Server farm configuration")
Figure 1: Server farm configuration
Public DNS servers
Internet
Router (Cisco Systems)
Load balancer (F5 Networks BIG-IP)
Load balancer (F5 Networks BIG-IP)
Front-end Web server farm (six servers)
SMTP and DNS server
Terminal services, debugging, and administration server
SQL Server server 1
SQL Server server 2
SQL Server server 3
SQL Server server 4
SAN unit (Hewlett Packard)
Active Directory domain controller 1
Active Directory domain controller 2
MOM server
Backup server (Veritas software)
Backup tape device
HTML transformation server
Imaging and installation server (Altiris deployment server)
Router (Cisco Systems)
Edge network
Windows SharePoint Services Storage
Windows SharePoint Services uses two different types of databases: content databases and configuration databases. Data in the lists and libraries on SharePoint sites are contained into the content database. The virtual server settings such as the mapping of site URL to physical storage location are stored into the configuration database. The configuration database can be on the same server as the content database, or on a dedicated server.
To test and validate Windows SharePoint Services server farm scalability, the Internet Platform and Operations group split the content data into two equal databases. Each content database was managed by an instance of SQL Server running on one of the two virtual servers. As well as testing scalability, this design also reduced the time of potential backups and restorations. Each content database was configured to support at least 90 GB of data. One of the content databases shared its virtual server with the configuration database. All sites in a server farm can connect to the same content and configuration databases, as shown in Figure 2.
.gif "Figure 2: Database configuration")
Figure 2: Database configuration
Each of the two back-end servers running SQL Server formed a virtual server. Both virtual servers were connected to the SAN as shown in Figure 3. A Hewlett-Packard SAN stored the data repository to help ensure high availability and data integrity by providing redundancy.
The servers were connected with a heartbeat local area network (LAN) that helped monitor server health.
.gif "Figure 3: SQL Server network")
Figure 3: SQL Server network
Four servers running SQL Server are configured as two Active/Passive-mode clusters by using Microsoft Cluster Service. The high-level design is shown in Table1.
Table 1 SQL Server clustering
Component |
Description |
|---|---|
Number of clusters |
Two |
Number of nodes per cluster |
Two – one active and one passive |
Number of virtual servers |
Two |
Operating system |
Microsoft Windows 2000 Advanced Server |
Operating system components |
Cluster Service |
Cluster administration software |
Cluster Administrator 5.0 Hewlett-Packard Secure Path Manager (SPM) 3.1 |
Hardware RAID Configuration
To increase system availability and help protect against data loss, the Internet Platform and Operations group used hardware RAID with the hardware controllers. The drive performance, type of server, and the data type to protect against were considerations in determining the level of hardware RAID protection needed. For example, RAID 1 provides adequate protection for Web Servers but RAID 1+0 is more suited for database storage requiring high performance and fault tolerance.
The following table lists the hard disk capacity and RAID type used by different servers in this configuration.
Table 2 Server farm hard disks
Servers |
Hard Disks |
RAID Type |
|---|---|---|
Front-end Web servers |
Two 18.2-GB mirrored hard disk drives |
RAID 1 |
Servers running Microsoft Active Directory directory service |
Three 18.2-GB logical drives for operating system, database, and logs |
RAID 1+0 |
Servers running SQL Server |
Two 18.2 GB hard disk drive |
RAID 1+0 |
Storage area network (SAN) unit* |
34 36-GB hard disk drives |
RAID 5 for data backups RAID 1+0 for SQL Server database files RAID 1 for Quorum drive |
Backup server |
Two 18.2-GB hard disk drives for operating system and software Four 18.2-GB hard disk drives for additional backups |
RAID 1+0 for operating system and software RAID 5 for backups |
*For details about the drives in the SAN, see the "SAN Configuration" section later in this paper.
Information about the different types of RAID types is widely available from resources such as Planning the Layout and RAID Level of Volumes in the Microsoft Windows Server 2003 Deployment Kit on Microsoft TechNet.
Server Configuration Details
This section provides information about the servers vital to storage and backup/restore servers running SQL Server and the backup tape device. For information about how other servers, such as the front-end Web servers and Active Directory servers, were configured, see Microsoft Windows SharePoint Services Hosting Configuration and Experience, the first paper in this series. See Table 2 in the "Hardware RAID Configuration" section of this paper for information about the hard disk drive configuration on these servers.
The front-end Web servers Veritas backup agent software and Hewlett-Packard Compaq Insight Manager software. The servers running SQL Server ran Hewlett-Packard SAN software and the Veritas backup agent software. The backup server ran Veritas Backup Exec 8.60 for Windows Servers.
The backup device was a Dell PowerVault 120T DLT4000 Tape Autoloader with the following specifications:
DLT Tape Device: Single Operation Tape Unit (Can perform one backup or restore operations at one time, not concurrent tape operations)
Capacity: Holds 7 tape drives
Tape Type: DLTtape IV 40/80GB Native/Compressed
SAN Storage
A SAN unit stored the data for the Internet Platform and Operations group deployment of Windows SharePoint Services. The unit comprised switches, two controllers, and three Proliant disk shelves, each of which housed fourteen 36.4-GB hard disk drives. The raw capacity of the SAN was 509.6 GB per shelf, or approximately 1.5 terabytes (TB) in total.
Figure 4 and Table 3 show how the SAN drives were allocated. Unused drives 105, 302, 402, 500, 504, 600, 601, and 604 are shown as white rectangles.
.gif "Figure 4: SAN disk layout")
Figure 4: SAN disk layout
Table 3 Key to SAN disks
.gif "wssipo06")
All drives except those used for backups (database or transaction logs) used RAID 1+0 (striping with mirroring). This maximizes input/output performance while still retaining recoverability for database files in the event of disk failures. Backup and log disks used RAID 5 for maximum performance and recoverability, and the quorum uses RAID 1 (mirroring).
Backing Up and Restoring Data
Scheduling
Administrators normally schedule backups during times of low activity usage. The majority of the external users accessing the hosted sites resided in North America, so backups were scheduled to coincide with non-business hours in North America. Administrators must also take into account how long each part of the backup will take. Table 4 shows the typical backup size and time needed for the four types of backups required for this deployment of Windows SharePoint Services.
Table 4 Backup times
Backup Type |
Description |
Size |
Time |
|---|---|---|---|
Active Directory server |
User account information stored on the servers running Active Directory directory service (daily) |
2.8 GB |
40 minutes |
SQL Server differential |
Changes to the databases since the last backup (daily) |
1 GB |
25 minutes |
SQL Server full |
Complete databases (weekly) |
8.5 GB |
2.5 hours |
Windows SharePoint Services log files |
See "Front-End Web Servers" section later in this paper for a list of logs (daily) |
20 MB |
10 minutes |
Backup times vary between deployments. By using the Dell PowerVault 120T DLT4000 Tape Autoloader, Veritas Backup Exec software, and the Hewlett-Packard DL360 backup server, the Internet Platform and Operations group achieved a backup and verification rate of approximately 3.4 GB per hour.
Table 5 shows the tape backup schedule for the deployment in local time for the servers Pacific Daylight Time. The intervals between backup jobs allowed time for jobs to complete before the next job started. The backup start times can be adjusted to accommodate future growth.
Table 5 Tape backup schedule
Job |
Sunday |
Monday |
Tuesday |
Wednesday |
Thursday |
Friday |
Saturday |
|---|---|---|---|---|---|---|---|
Active Directory Server |
22:00 |
22:00 |
22:00 |
22:00 |
22:00 |
22:00 |
22:00 |
SQL Server -Differential |
01:00 |
01:00 |
01:00 |
01:00 |
01:00 |
01:00 |
01:00 |
SQL Server - Full |
01:00 |
||||||
WSS Log Files |
05:00 |
05:00 |
05:00 |
05:00 |
05:00 |
05:00 |
05:00 |
Backing Up and Restoring Servers
Data on three types of servers were backed up: front-end Web servers, Active Directory servers, and servers running SQL Server.
Front-End Web Servers
The data backed up from front-end Web servers was archived for analysis and long-term off-line reference, and never intended to be restored to the server. The following log files were backed up from these servers:
Internet Information Services (IIS) logs C:\Winnt\System32\Logfiles\W3svc1\*.log
Usage analysis log C:\Windows\System32\LogFiles\STS\ if usage analysis is enabled
Other Windows SharePoint Services logs STSAdm.log and OWSTimer.log from the C:\Documents and Settings\Windows_SharePoint_Services_Administrator_Account\Local Settings\Temp directory.
Active Directory Servers
Backing Up Active Directory Servers
In this deployment, user accounts were replicated between the two peer Active Directory domain controller servers. In addition, the system state data on each Active Directory server was first saved to disk and then picked up nightly by the backup tape device. To avoid possible timing, performance, and replication issues, the system state was saved to disk at different start times for each Active Directory server. For example, the system state for the first server was saved at 19:30, the system state for the second server was saved at 21:30, and then the backup of the system states and user accounts began at 22:00. The Internet Platform and Operations Group used the Scheduled Tasks tool in Microsoft Windows Server 2003 to schedule these tasks.
System state data on the Active Directory serves included Active Directory and all other system components and services on which Active Directory is dependent, the system startup files, system registry, file replication service (the SYSVOL directory), and Domain Name System (DNS, if it is installed). The DNS data includes DNS zone information that is integrated with Active Directory. Active Directory includes the following files:
Ntds.dit The Active Directory database.
Edb.chk The checkpoint file.
Edb*.log The transaction logs
Res1.log and Res2.log Reserved transaction logs
Note: You must be a member of the local Administrators group on the Active Directory servers to back up Active Directory data.
Restoring Active Directory Servers
If data on disks is destroyed, an administrator can copy the data from tapes to a newly allocated hard disk, and then restore the Active Directory by using either a normal (non-authoritative) restore or an authoritative restore.
The normal restore allows the Active Directory server that was not restored to replicate changes to the restored Active Directory server. The restored server therefore reflects changes made after the backup that was used to restore it. The authoritative restore through the Ntdsutil utility prevents replication between servers for all items created before the restore. For example, you may only want to restore the users or OUs (Organizational Units) that are accidentally deleted from Active Directory, so that the deleted objects are recovered and replicated.
For more information about the different kinds of restores and restoring Active Directory servers, see Windows Server 2003 and Active Directory documentation.
Servers Running SQL Server
Backing Up Servers Running SQL Server
Windows SharePoint Services content and configuration databases and the MSSQL database were backed up first to disk by using SQL Server 200 Enterprise Manager, and then to the tape device by using Veritas Backup Exec software. Separate SAN partitions were configured for the data and backup drives.
Because the Internet Platform and Operations deployment of Windows SharePoint Services hosted test sites for beta users and the deployment was designed with enough fault tolerance to survive during some disk and server failures, the Internet Platform and Operations group used the SQL Server simple recovery model. In the SQL Server simple recovery model, log files are not backed up. For business-critical deployments, administrators should use full recovery mode. For information about the different recovery modes, see SQL Server documentation.
SQL Server Enterprise Manager allows administrators to schedule backups for each database in the system. In SQL Server Enterprise Manager, the Internet Platform and Operations Group entered backup tasks for daily and weekly backups of each database on both virtual servers.
The following diagram illustrates the scheduled job activity on the first virtual server. Similarly, on the second virtual server, the content and system databases are written to disk before they are copied to tape.
SQL Server Agent in SQL Server Enterprise Manager shows the status of the disk backup jobs in Figure 6, so administrators can determine whether backups are successful.
.gif "Figure 6: SQL Server Enterprise Manager")
Figure 6: SQL Server Enterprise Manager
Table 6 Disk backup schedule for SQL Server databases
Database |
Server |
Type |
Sun |
Mon |
Tues |
Wed |
Thurs |
Fri |
Sat |
|---|---|---|---|---|---|---|---|---|---|
MSSQL |
Virtual servers 1 and 2 |
Full |
19:00 |
19:00 |
19:00 |
19:00 |
19:00 |
19:00 |
19:00 |
Configuration database |
Virtual server 1 |
Full |
19:30 |
19:30 |
19:30 |
19:30 |
19:30 |
19:30 |
19:30 |
Content database 1 |
Virtual server 1 |
Diff. Full |
20:30 |
20:30 |
20:30 |
20:30 |
20:30 |
20:30 |
20:30 |
Content database 2 |
Virtual server 2 |
Diff. Full |
20:30 |
20:30 |
20:30 |
20:30 |
20:30 |
20:30 |
20:30 |
Restoring Servers Running SQL Server
SQL Server provides command-line options and graphical user interface options that administrators can use to restore data to servers. For more information, see SQL Server documentation.
Backup Failures
Administrators should choose backup software that alerts them if a backup job does not run or stops prematurely. Software should also track jobs that run successfully. Refer to your backup software documentation for specific setup and configuration.
Summary
Administrators deploying Windows SharePoint Services to host customer sites can build on the experience that the Microsoft Internet Platform and Operations group when they deployed Windows SharePoint Services (Beta) for a similar use. From choosing servers to setting up customer sites, administrators can be confident that someone has been through this before. For more information about the entire environment of the Windows SharePoint Services (Beta) hosting deployment, see the other white papers in this series.
Related Links
See the following resources for further information:
Windows SharePoint Services Hosting Configuration and Experience at https://go.microsoft.com/fwlink/?linkid=18323&clcid=0x409
Windows SharePoint Services Network and Load Balancing Design at https://go.microsoft.com/fwlink/?linkid=18325&clcid=0x409
Windows SharePoint Services Monitoring and Instrumentation at https://go.microsoft.com/fwlink/?linkid=18326&clcid=0x409
Microsoft Windows SharePoint Services Administrators Guide at https://go.microsoft.com/fwlink/?linkid=18327&clcid=0x409
For the latest information about Windows Server 2003, see the Windows Server 2003 Web site at https://www.microsoft.com/windowsserver2003/default.mspx.