Investigating Alert Storms

 

Updated: May 13, 2016

Applies To: System Center 2012 R2 Operations Manager, System Center 2012 - Operations Manager, System Center 2012 SP1 - Operations Manager

A large and sudden increase in the number of alerts is called an alert storm. An alert storm can be a symptom of massive changes of some kind within your management group, such as the catastrophic failure of networks. An alert storm can also be a symptom of configuration issues within System Center 2012 – Operations Manager.

Installing new or updated management packs can give rise to an alert storm. Monitors in a management pack begin working as soon as the management pack has been imported. Use best practices in importing management packs to minimize alert storms.

Finding Alert Storms

For general, real-time monitoring of alerts, use the Active Alerts view. Make sure Scope is not active and hiding alerts. For more information, see How to Change Scope.

Check for large numbers of alerts when your network undergoes changes. Monitor closely when you install a new management pack.

Operations Manager offers reports that can be useful in identifying alert storms. From an Operations console with access to a reporting server, look at the Microsoft Generic Report Library. The reports Most Common Alerts and Most Common Events help identify high-volume alerts.

Modifying Monitors and Rules

If you are getting a large number of alerts that do not point to issues in your managed systems, you need to modify the monitors or rules that create those alerts.

View active alert details in the Monitoring workspace. Alert Details specifies the monitor or rule for an alert.

Modify the monitor using overrides. The procedure for overriding rules is the same as for monitors. See how your overrides affect the amount of alerts and continue to fine-tune the monitors as necessary. For more information, see Tuning Monitoring by Using Targeting and Overrides.

About Suppressed Alerts

Rules offer the option of suppressing duplicate alerts. A suppressed alert is not displayed in the Operations console. Each suppressed alert increments the repeat count for the alert that is displayed. You can examine the repeat count in the properties for an alert.

Operations Manager suppresses only duplicate alerts as defined by the alert suppression criteria. Fields stated in the suppression criteria must be identical for the alert to be considered a duplicate and suppressed. An alert must be created by the same rule and be unresolved to be considered a duplicate.