Maintaining and Monitoring Sites

Article
12/08/2008

Published : September 1, 2004

After the site is fully deployed, you should start performing the site monitoring and maintenance tasks according to the maintenance plan that you developed.

While monitoring the sites, look for signs that indicate problems that require intervention. For example:

File backlog on site servers and site systems.
Status messages that indicate an error or a problem.
Failing intrasite communication.
Error and warning messages in the system’s event log on the most important servers.

Note
When SMS is configured to write status messages to the system’s event log, SMS error status messages are written as information events rather than error events.
Error and warning messages in the Microsoft SQL Server™ error log.
Sites or clients that have not reported in a long time.
Signs of hardware failure.

There are various reasons that can cause SMS sites to fail. To minimize the risk of failure, if monitoring tasks reveal any signs of problems, investigate the source of the problem and repair it.

SMS sites can also fail because of hardware problems. Even high-quality hardware occasionally fails. Sometimes, hardware fails gradually and there might be early signs of failure. As soon as you notice any signs of unreliable behavior on site server’s hardware, swap the computer of the site server. Replacing site server hardware before it completely fails is an important step in preventing site failure.

Configuring and Running Predefined and Custom Maintenance Tasks

Most predefined SMS maintenance tasks are enabled and configured by default; however, you can change those settings as needed in your organization. Predefined and custom maintenance tasks log their activity to the SMS_SQL_Monitor log file (SMSdbmon.log).

To configure and run a predefined maintenance task

In the SMS Administrator console, navigate to Tasks:

Systems Management Server Site Database Site Hierarchy <site code> - <site name> Site Settings Site Maintenance Tasks

In the details pane, right-click the task that you want to run and click Properties.
In the task Properties dialog box, enable and configure the task. To minimize interference with the site operation, set the time period to off-peak hours of the site. The time period* *is the time interval in which the task can run. It is defined by the Start after and Latest start time specified in the task properties dialog box.
Click OK to save your settings. The task will now run according to its schedule.

If a task fails to run at first attempt, the SMS_SQL_Monitor service attempts to rerun the task until either the task runs successfully, or until the time period in which the task can run has passed.

To use custom maintenance tasks, you must first create them, and then configure them as needed.

To create and configure a custom maintenance task

Caution
Custom SQL commands are very powerful, because they directly manipulate the SMS site database. Improper SQL commands can corrupt the SMS site database. You should secure the SMS site database by limiting the Manage SQL Commands SMS object security right to the appropriate SMS administrators.

In the SMS Administrator console, navigate to SQL Commands.

Systems Management Server Site Database Site Hierarchy <site code>- <site name> Site Settings Site Maintenance SQL Commands

Right-click SQL Commands, select New, and then select SQL Command.
In the SQL Command Properties dialog box, enter a name and configure the task:
1. Specify a SQL command. You can specify either a single line (255 characters) SQL command, or the name of a stored procedure that contains multiple SQL commands.
2. Specify a log file name if you want to review the results after the SQL command runs.
3. Specify a schedule for the SQL command.
  
  Note
  Before you schedule a custom maintenance task, ensure that the SQL command is valid by testing it in SQL Query Analyzer.

Daily, Weekly, and Periodic Tasks - Best Practices

In this section, there is a recommended list of daily site maintenance and monitoring tasks. Different organizations might find it is necessary to add or remove items from the list to better maintain their sites. Adjust the frequency of these tasks to fit the capacity of the site and the needs of your organization. Depending on the results of the monitoring tasks, you might need to perform additional tasks to handle problems in the site.

For security related maintenance tasks, see Chapter 5, “Understanding SMS Security,” in the Microsoft Systems Management Server 2003 Concepts, Planning, and Deployment Guide.

Use the Maintenance Tasks Tracking Spreadsheet to track the ongoing execution of the planned daily, weekly, and periodic site maintenance tasks.

MainTasksTracking.xls

Daily Maintenance and Monitoring Tasks

To follow site maintenance best practices, schedule the following predefined maintenance tasks to run on a daily basis:

Delete Aged Status Messages.
Back up SMS Site Server.

To follow site maintenance best practices, perform the following monitoring tasks on a daily basis. If there is any indication of a problem, isolate and repair the problem to ensure that the site remains healthy.

Check SMS site database status.
Check site server status.
Check site systems status.
Check client status.
Check the operating system event log.
Check the SQL Server error log.
Check system performance.
Check SMS system folders.

Weekly Maintenance and Monitoring Tasks

To follow site maintenance best practices, perform the following maintenance tasks on a weekly basis. For SMS predefined tasks, and other automated tasks, schedule the task to run on a weekly basis:

Rebuild indexes.
Monitor keys.
Delete aged inventory history.
Delete aged discovery data.
Delete aged collected files.
Delete aged software metering data.
Delete aged software metering summary data.
Delete Inactive Client Discovery Data
Delete Obsolete Client Discovery Data
Summarize software metering file usage data.
Summarize software metering monthly usage data.
Delete unnecessary files.
Delete unnecessary SMS objects.
Produce and distribute end-user reports.
Run disk defragmentation tools.
Back up application, security, and system event logs.

To follow site maintenance best practices, perform the following monitoring tasks on a weekly basis. If there is any indication of a problem, isolate and repair the problem, to ensure that the site remains healthy.

Check SMS site database available space.
Check available disk space.
Verify that scheduled maintenance tasks are running.

Periodic Maintenance and Monitoring Tasks- Best Practices

To best maintain your system, perform the following maintenance tasks periodically. To optimize and protect your SMS sites, determine the frequency to schedule tasks, based on the capacity of the site and the needs of your organization. Then, perform them according to the schedule.

Back up account data.
Change accounts and passwords.
Check network performance.
Review the security plan.
Review the maintenance plan.
Perform recovery test in a test lab.

To best maintain your system, perform the following monitoring tasks periodically. To optimize and protect your SMS sites, determine the frequency to schedule these tasks, based on the capacity of the site and the needs of your organization. Then, perform them according to the schedule. If there is any indication of a problem, isolate and repair the problem to ensure that the site remains healthy.

Check hardware.
Check site’s overall health.
Check the backup snapshot.

Changing Site Configuration, Hardware, and Infrastructure

After SMS is deployed, the organization is likely to experience changes over time. You might need to make changes to the SMS hierarchy, to hardware, or you might need to adjust SMS to changes made to the organization’s infrastructure. This section describes some of the possible changes, and how they impact SMS.

Accommodating Changes

Before making changes, it is recommended that you follow these best practices to accommodate changes to the site.

Test changes in the test lab

Before implementing changes to the entire hierarchy, it is recommended that you first test these changes in a test lab. When testing the proposed change in the test lab, also perform a site backup and recovery test. If the proposed change requires an adjustment to the site’s backup and recovery plan, then make the necessary changes.

For more information about the test lab, see Scenarios and Procedures for Microsoft Systems Management Server 2003: Planning and Deployment on Microsoft TechNet.

Update the test lab to reflect changes

If you are using a test lab, then you need to maintain it so that it continues to correctly represent your production environment. If you make changes in the production environment, such as adding new client platforms, changing your network infrastructure, or making changes to feature sets, then you must make the corresponding changes in the test lab.

As the proposed changes are implemented in the deployment environment, update the test lab as needed to reflect those changes.

For more information about the test lab see Scenarios and Procedures for Microsoft Systems Management Server 2003: Planning and Deployment on Microsoft TechNet.

Update respective documents

To correctly recover a site, you must have all site server configuration data available. Ensuring that this data is always up-to-date helps you in case a backup snapshot is not available for a recovery operation, or in the event that there is no staff familiar with the hierarchy deployment.

Whenever information such as the following changes, you need to update the respective document in which the information is stored. This is important so that you can use this information during a recovery operation:

Hierarchy design, including data such as parent-child relationships, primary and secondary site locations, and the computers on which SQL Server is running.
Site configuration for each SMS site in the hierarchy, including data, such as feature settings and component settings.

Security information, such as user names and passwords for client connection accounts, service accounts, and network access accounts. This includes changes to accounts such as the Client Push Installation account and the Site Address account.

You can use the Network Trace tool to create and print SMS network site views to help you maintain up-to-date diagrams of site layout. For more information about using the Network Trace tool, see Chapter 10, “Maintaining and Monitoring the Network,” in the Microsoft Systems Management Server 2003 Operations Guide.

Update recovery tasks

If you store prepared recovery tasks from the Recovery Expert, then whenever there are changes to the site configuration, you should rerun the Recovery Expert with updated answers. Back up the updated document, and store in a safe location.

Unsupported Configuration Changes

Not every site configuration change is supported. For example, the following changes are not supported:

Changing the drive letter on which SMS is installed.
Changing the site code.
Changing the site name.
Changing the computer name of the site server.
Changing the parent site of a secondary site.
Moving the SMS site database from a remote server to the site server.
Promoting a Microsoft Windows® 2000 member server that is an SMS site server to be a domain controller.
Renaming the domain where SMS sites are deployed.

Site Configuration and Hardware Changes

After setting up and configuring an SMS site, you might need to change the initial site configuration in order to accommodate growth or other changes in your organization. Other than configuration changes, it might be necessary to replace the computer of an SMS server. Not all changes are supported, and to accomplish some changes, you must carefully follow a prescribed procedure.

Some of the supported site configuration and hardware changes include:

Adding a child site post deployment
Moving the SMS site database
Rebuilding the computer of a remote SMS site database server
Resetting the site by running SMS site reset
Swapping the computer of a site server
Upgrading the operating system of the site server and other site systems
Uninstalling the site server, or other site components

For information about client related maintenance operations, such as replacing a Legacy Client with an Advanced Client, see Chapter 17, “Discovering Resources and Deploying Clients,” in the Microsoft Systems Management Server 2003 Concepts, Planning, and Deployment Guide.

Infrastructure Changes

Some maintenance tasks are associated with certain infrastructure changes in the organization, basically un-related to SMS. However, those changes might require some adjustment to SMS. For example, changing the Active Directory® site boundaries is a change that might require that you adjust the SMS site configuration in order to accommodate the new change.

This section describes changes in the organization that affect SMS, and the tasks that you need to perform to accommodate those changes:

Changing Active Directory site boundaries
Changing TCP/IP subnets
Changing SQL Server configuration
Changing Hardware or Software on the Site Server
Changing Wide Area Network infrastructure

Adding a Child Site Post Deployment

During the deployment phase, the sites that you attach to each other are not fully active, and therefore still have only a limited amount of data and objects. However, after you have deployed the hierarchy, and the sites are now fully functional, there might be a need to attach additional sites. In this scenario, it might be important to know how do the objects from the new child site, and from the parent site behave after the attachment.

After attaching one site to another, the important thing that happens is that data starts to flow from the child site to its parent site and from the parent site to its child site.

Objects, such as software distribution related objects, and collections, are replicated to the child site, and can be used at the child site. However, those replicated objects are locked at the child site. The administrator at the child site cannot modify inherited objects. For example, the administrator at the child site can use an inherited collection definition to create a collection of computers at the child site.

Predefined objects, such as predefined collections that are inherited from a parent site, overwrite the corresponding objects at the child site. If you modified any of those objects at a site that you plan to attach to another site, ensure that the changes are backed up, or plan to make those changes again after the attachment process is complete.

Data such as status messages and inventory is replicated from child sites to their parent site, and then from the parent site to its parent site. This ensures that, eventually, the central site contains data from all the clients of all sites in the hierarchy.

Moving the SMS Site Database

To accommodate growth and other changes in your organization, you might want to move the SMS site database to a remote database server. Moving the SMS site database to a remote server is performed in the same manner for the following two scenarios:

Moving a local SMS site database to a remote server
Moving the SMS site database from one remote server to another remote server

When moving the SMS site database server, SMS moves the SMS Provider to the same server that the SMS site database is moving to.

Caution
The following procedure can corrupt the SMS site database, therefore, it is recommended that only experienced SQL Server users perform the following procedure.

To move SQL Server to a remote server

Install SQL Server on the remote server.
Configure the new SQL Server database security so it is similar to the old SQL Server security configuration. Configure accounts and permissions required for SMS to manage SQL Server.
Close all SMS Administrator console sessions and ensure that no Administrator console session is opened until the end of this procedure. Opening Administrator console session will restart the WINMGMT service (see step number 4), which should not be running during this procedure.
Stop the following SMS services:

On the SMS Provider:
- WINMGMT
On the site server:
- SMS_SITE_COMPONENT_MANAGER
- SMS_EXECUTIVE
On the old database server:
- SMS_SQL_MONITOR_<SiteServerName>
Back up the SMS site database from the old SQL Server database by using the SQL Server Backup Database tool.
Restore the SMS site database to the new SQL Server database by using the SQL Server Restore Database tool.
On the site server, run the SMS Setup Wizard:

On the Setup Options page, select Modify or reset the current installation.
1. On the Database Modification page, specify the new database server name and the appropriate database name.
2. Complete the SMS Setup Wizard.
Run Site upgrade from the SMS CD.
You can now delete the SMS site database from the old SQL Server database, or disconnect the old database server (if it is on a remote computer).

Note
While using the new SMS site database server, SMS generates error status messages as it continues to attempt to access the old SQL Server installation. You can safely ignore those status messages.

Rebuilding the Computer of a Remote SMS Site Database Server

This section provides information about moving a healthy and a functioning remote SMS 2003 site database server from one computer to another. This may be necessary if the remote site database server is exhibiting circumstances, such as the following, and you need to rebuild the computer:

The computer is exhibiting signs of imminent hardware failure.
The operating system is failing and it is necessary to reinstall the operating system.
SQL Server is failing and it is necessary to reinstall SQL Server.

You can follow this procedure for any other restoration operation of a remote site database server. As long as the SMS site data on the remote database server is intact, you can treat the operation as a hardware swap operation.

To rebuild the computer of a remote site database server

This scenario assumes that the operating system, SQL Server, and data files are each on a separate drive on the database server.

On all sites sharing the SQL Server that is being swapped, you must stop the SMS_SITE_COMPONENT_MANAGER service and the SMS_EXECUTIVE service in order to prevent any process from accessing the databases.

Stop and disable SMS services at each site as follows:
1. Stop the SMS_SITE_COMPONENT_MANAGER and SMS_EXECUTIVE services on the site server.
2. Stop the SMS_SQL_MONITOR service on the SMS site database server.
3. Stop the Windows Management Instrumentation service on the site server, and on the SMS site database server.
Disable these services to prevent them from restarting. To disable a service, right-click the component name, and then click Properties. From the Startup type shortcut menu on the component Properties page, select Disabled.
Rebuild the operating system on the failing computer, or use a different computer with an intact operating system.
Reinstall SQL Server. Restore the master, model, and msdb system database from the last backup to recover SQL Server permissions and definitions.
Recover the SMS database in place without restore (see SQL Server documentation about recovering and reconnecting to existing data files in place). The data is now intact and still synced up with the site
Inspect databases to ensure that they can be accessed and appear to contain valid data.
Select the Modify option to run SMS Setup and perform a site reset..

Resetting a Site by Running SMS Site Reset

SMS site reset stops the core SMS site services, removes them, and then reinstalls them. By using SMS site reset, you can do the following:

Repair a damaged site server.
Make account or password changes effective.
Force the site to use a different SQL Server database name or a server running SQL Server, or a different SQL Server security mode.

If there has been a change to the accounts used by the SMS components, site reset ensures the account details used by the SMS components are correct.

Note
To run site reset on a secondary site, you must start it from the product CD rather then from the Start menu.

The exact effect of site reset depends on how you initiate it and which options you chose. For more information about site reset, see Chapter 5, “Understanding SMS Security,” in the Microsoft Systems Management Server 2003 Concepts, Planning, and Deployment Guide.

Swapping the Computer of a Site Server

Even high-quality hardware occasionally fails. Sometimes, it fails gradually, and there might be early signs. Replacing site server hardware before it completely fails is an important step in preventing site failure.

As soon as you notice any signs of unreliable behavior on site server’s hardware, replace the hardware. To properly replace site hardware, you must use the Recovery Expert, which is a recovery tool.

This section provides information about moving a healthy and a functioning SMS 2003 site server, in a primary or secondary site, from one computer to another. This might be necessary if the current site server computer is exhibiting signs of imminent hardware failure, or if the lease expires on a server hosting SMS roles.

Swapping the site server computer consists of the following basic steps:

Preparing for the hardware swap.
Building the new site server.
Restoring the SMS site data.
Repairing and synchronizing the data.
Verifying the functionality of the new site server.

A site server computer swap process is somewhat similar to a site recovery process. However, because the current site server is properly functioning, you can plan and completely control the operation. The operation is significantly simpler then a recovery operation, and no data is expected to be lost.

Because a hardware swap operation is similar to a site recovery operation, you can use the Recovery Expert to perform this operation. The Recovery Expert is designed to handle this scenario, and when you run it, it provides the appropriate tasks for a hardware swap operation.

To swap the computer of a site server by using the Recovery Expert

Set up the Recovery Expert Web site if necessary.
Run the Recovery Expert.
Answer Yes to the question Do you need to move a site server to different hardware?
Answer Yes to the question Are you recovering the site because you need to re-install the operating system?
Perform the prescribed tasks in the prescribed order. Use the SMS Site Repair Wizard and other repair tools as instructed by the Recovery Expert.

Upgrading the Operating System of the Site Server and Other Site Systems

Upgrading the operating system of SMS site systems is supported, and is usually successful and without any interference with SMS functionality. However, in some upgrade scenarios, some intervention is required to ensure proper operation of SMS after the operating system upgrade.

Upgrading the Operating System of the Site Server

After upgrading the site server from Windows 2000 to Windows Server™ 2003, the IIS (Internet Information Services) service is disabled. This prevents the management point, the server locator point and the reporting point from working properly. To resume regular operation of those site systems after the operating system upgrade, you must manually enable the IIS service.

Note
If an operating system upgrade fails, and the computer needs to be restored, then you must recover the computer using the original operating system and the site’s backup snapshot. Then you can attempt to upgrade the operating system again.

Upgrading the Operating System of the SQL Server

Before updating or upgrading the operating system of the SQL Server, you must stop SMS services.

Stop the SMS_Site_Component_Manager service first, and then stop all other SMS services and Windows Management Instrumentation (WMI) services for the SMS Provider.

Stop those services by entering the following command in a command prompt:

Net stop <service>

Upgrading the Operating System of a Server Locator Point or a Management Point

When upgrading the operating system of a server locator point or management point from Windows 2000 to Windows .Net Server, you must reset the site after the upgrade is complete.

If you do not reset the site after upgrading a management point, the management point will not be able to perform its basic functions, such as reporting client installation status, checking for policies, and storing data.

If you do not reset the site after upgrading a server locator point, the server locator point cannot perform its basic functions, such as determining client assignment during client installation.

Uninstalling the Site Server, or Other Site Components

As an organization changes, there might be a need to uninstall a site server, a site system, or some or all the clients.

Uninstalling Clients

For information about removing Advanced Clients and Legacy Clients, see the ”Removing and Repairing SMS Client Software” section, in Scenarios and Procedures for Microsoft Systems Management Server 2003: Planning and Deployment on Microsoft TechNet.

Uninstalling Primary and Secondary Sites

To properly uninstall a primary or secondary site server, you must first remove the Advanced Clients, Legacy Clients, and local and remote site systems from the site.

For detailed information about uninstalling sites, see the ”Removing a Primary Site” and ”Removing a Secondary Site” sections in Scenarios and Procedures for Microsoft Systems Management Server 2003: Planning and Deployment on Microsoft TechNet.

Important
When using an Administrator console at a parent site to uninstall a child secondary site, the secondary site’s object might not be removed from Active Directory. If the site’s SMS-Site-<site_code> object remains in Active Directory, than you need to manually remove it.

Uninstalling a Management Point

A management point must be properly removed to ensure that its respective objects (SMS-MP-<site_code>-<computer_name>) are deleted from Active Directory. If the objects of a management point that no longer exists remain in Active Directory, that management point continues to appear as a valid management point to clients. Clients might then select it as their resident management point.

Management point objects might be left in Active Directory if, for example, you uninstall a site server without first removing the local management point, or you remove a remote management point while the site server is not functioning properly.

If management point objects are left in Active Directory for a non-existent management point, you should manually remove those objects from Active Directory.

Uninstalling a Reporting Point

When uninstalling a reporting point, all custom reports in the Reporting Point\Supplemental are removed. In order to keep those reports, you must back them up before disabling the reporting point.

Changing Active Directory Site Boundaries

SMS site boundaries settings can be based on Active Directory sites. In this case, if there are changes to Active Directory sites in your organization, then the SMS administrator needs to be notified about the change. The SMS administrator must adjust any SMS site boundaries that are based on Active Directory sites that have changed.

Changing TCP/IP Subnets

Whenever you make changes to TCP/IP subnets in the infrastructure of your organization, such as adding or removing a TCP/IP subnet, you need to adjust SMS site boundaries to reflect the change. This is necessary to ensure that clients are assigned correctly to SMS sites.

Clients are assigned to sites according to the site’s site boundaries and roaming boundaries. Because those boundaries can be based on IP Subnets, making changes to IP Subnets in the site’s site boundaries, or in the site’s roaming boundaries, changes client site assignments. Changes might leave clients unassigned, causing them to uninstall their SMS client components. To ensure that clients are not uninstalled when adding TCP/IP subnets, you must add the new IP subnet as a site boundary in the site’s properties before the new physical subnet is connected.

Changing SQL Server Configuration

To properly recreate database devices, and to configure SQL Server before restoring a failed site, you must have the SQL Server configuration information (such as user connections, locks, and memory) and complete device allocation information (such as device size and type).

Run SMSSQLinfo.bat from the SMS\Bin\i386 folder to back up SQL Server configuration information whenever you make changes. For more information, see the SQL Server Help.

Changing Hardware or Software on the Site Server

You should have a system recovery solution for SMS site servers. This allows you to recover the site server computer if it starts failing. For site servers running any of the Windows 2000 Server family, use an Emergency Repair Disk. For site servers running Microsoft Windows Server™ 2003, Standard Edition; Windows Server 2003, Enterprise Edition; or Windows Server 2003, Datacenter Edition, use the system’s Automated System Recovery feature.

Update the system recovery data every time you make significant changes to the system’s hardware or software. Changes that require an update include changes to the partition structure, changes to device drivers, and installing new applications.

Changing Wide Area Network Infrastructure

Whenever changes are made to the wide area network (WAN) infrastructure used in the hierarchy, check addresses and senders settings throughout the hierarchy, as appropriate. Ensure that they correctly reflect the new WAN infrastructure. Analyze the changes and determine if you need to add new SMS sites in order to accommodate the WAN infrastructure.