Troubleshooting State Message Backlogs
Applies To: System Center Configuration Manager 2007, System Center Configuration Manager 2007 R2, System Center Configuration Manager 2007 R3, System Center Configuration Manager 2007 SP1, System Center Configuration Manager 2007 SP2
Configuration Manager 2007 clients send state messages to report important changes of state for some Configuration Manager 2007 features, including client deployment and other client communication, software updates, Network Access Protection (NAP), asset intelligence, and desired configuration management.
Some state messages sent by clients are sent to the fallback status point, where they are processed and forwarded to the management point site system, while others are sent directly to the management point. The amount of state message traffic being sent from the fallback status point to the management point site system computer can be throttled to control the number of state messages processed at the same location. For more information about state message throttle settings, see Determine If You Need to Configure Throttle Settings for the Fallback Status Point in Configuration Manager.
State messages processed by the fallback status point, as well as state messages sent directly from Configuration Manager 2007 clients to management point site systems, are forwarded to the state system inbox (Statsys.box\incoming) on the site server computer for a site. You can monitor the files in the state system inbox or use the Configuration Manager 2007 Management Pack for Operations Manager 2007 to monitor the SMS Inboxes counter. If you have child sites sending state messages, you can monitor the site server's Replmgr.box\incoming inbox for backlogs of state messages.
While it is normal to have some state message processing backlog, it is important to plan for the impact of site server state message processing when performing activities that generate large numbers of state messages. For example, because clients send multiple state messages during client installation, it is important to restrict the number of client upgrades during periods of high software update activities. When Configuration Manager 2007 clients install multiple software updates, each client sends multiple state messages for each individual software update installation. The impact of both software update and client installation activities can quickly cause a state message processing backlog on the site server computer.
|The only way to view consolidated client state message information is through Configuration Manager 2007 state message reports generated by the reporting point site system. A large or persistent state message processing backlog can prevent Configuration Manager 2007 from reporting critical information. For more information about which reports use the fallback status point to relay state messages from clients, see About Reports for Configuration Manager Clients.|
The following activities generate significant numbers of state messages:
Initial software update scans after client installation
Software update scans during the regular security update release on the second Tuesday of the month
Asset Intelligence Client Access License (CAL) data collection
Desired configuration management processes
To reduce the chances of developing a large backlog, follow these guidelines:
If you are using the Configuration Manager 2007 software updates feature, do not deploy clients during major software update deployments.
Limit the number of clients deployed at any one time. For example, use the Client Push Installation Wizard instead of enabling the site-wide client push installation method.
Do not configure the Asset Intelligence CALCollectionFrequencyDays more frequently than the default (once per week), and set the CALCollectionType to only the minimum data required. For more information, see How to Enable Asset Intelligence.
If using the desired configuration management feature, limit the number of baselines per client and the number of configuration items per baseline.
If a significant state message backlog develops, it can be difficult for Configuration Manager 2007 to recover. If you determine that you have a significant backlog and it does not resolve itself over time, take action to reduce the number of state messages generated. For example, disable the Software Updates Client Agent or the Desired Configuration Management Client Agent until the backlog subsides, or disable Asset Intelligence CAL data collection, or temporarily stop client deployment.
Each state message sent by clients is identified by a serial number as it is sent by the client. The site server keeps track of the next expected state message serial number for each client. State system resynchronization requests are sent by the site server to clients when missing state messages are detected or state messages are processed out of order. State messages can be missing for a variety of reasons, including recovery of a failed site from backup, migration of a client (or an entire site) from one part of the hierarchy to another, unprocessed state messages that were rejected during a migration from mixed mode to native mode, clients being offline longer than the message send time-out period, and other miscellaneous communication failures.
If a site has become heavily backlogged, the site server might not have processed the client state message resynchronization complete state message when the next state message resynchronization check cycle occurs. If the resynchronization completion state message is not processed after three days, a state message resynchronization request will be reissued to the client—even if the client’s resynchronization complete message has already been sent but not processed yet.
|The state message serial number tracking is performed at every site in the hierarchy. If a client is assigned to a child site of the site that discovers the missing state message serial number, the resynchronization request is replicated to the child site and acted upon by the child site. If the client is assigned to the current site, the resynchronization request policy is generated immediately.|
When a client receives the resynchronization policy, it resends all the items in its cache (which represents the latest message for each topic type and topic ID) in a single file sent to the server. These messages are renumbered according to the current serial number so that they represent new messages and not replacement messages. Along with the messages from the client cache, a resynchronization complete state message (TopicType=700, StateID=2) is sent. At this point, the resynchronization is complete, the client should not have any missing state messages, and the current client state in the database should be complete.
Either of the following Microsoft SQL Server queries can be used to check for outstanding resynchronization requests issued by the current site or any of its child sites.
|Times are in UTC.|
select count(*) from SR_ClientMessageStats where LastResyncIssuedTime is not null and (LastResyncIssuedTime > LastResyncClientCompletedTime)
select count(*) from SR_ClientMessageStats where LastResyncIssuedTime is not null and LastResyncClientCompletedTime is null
For additional information, see Configuration Manager 2007 Information and Support.
To contact the documentation team, email SMSdocs@microsoft.com.