Delivery Failure Delivery-StoreDriver 5.6.0 happened over last 5 minutes - Yellow(>5)

 

Applies to: Operations Manager Management Pack for Exchange 2010

Topic Last Modified: 2011-08-02

The Microsoft Exchange Server 2010 Management Pack for System Center Operations Manager includes a performance data collection engine that is used to query performance counter objects on computers running Exchange 2010. For this Operations Manager rule, data is collected by using the performance counter specified in the Details table.

To review the value of the performance counter that generated this alert, in Operations Manager, double-click this alert, and then click the General tab. Review the description of the alert that includes the variables specific to your environment.

Details

Product Name

Exchange

Product Version

14.0 (Exchange 2010)

Object Name

MSExchangeTransport Delivery Failures

Counter Name

Delivery_StoreDriver_5_6_0

Instance Name

<All>

Sample Interval

60

Server Role

Ex14. Common

Warning Threshold

5

Rule Path

Microsoft Exchange Server/Exchange 2010/Common Components/Hub Transport and Edge Transport/Transport

Rule Name

Delivery Failure Delivery-StoreDriver 5.6.0 happened over last 5 minutes - Yellow(>5).

Explanation

This event indicates that the Store Driver component has experienced more than five message delivery failures in the previous five minutes. In this scenario, incoming messages may not be delivered to mailboxes and outgoing messages may not be delivered to transport queues for processing.

The Store Driver component (Microsoft.Exchange.StoreDriver.dll) runs on computers that are running the Hub Transport server role or the Edge Transport server role. Store Driver is the Microsoft Exchange Server 2010 Transport service component that is responsible for communicating with the local store for mail submissions and local message delivery. The Store Driver has the following main functions:

  • The driver delivers messages from the Transport service to the information store.

  • The driver moves newly submitted messages from the information store to the Transport service so that the Transport service can process the messages.

Exchange 2010 introduces a new extensibility model for Store Driver. This model resembles that of the SMTP and Categorizer (MExRuntime) components. The Store Driver design change helps make it possible to move all code that is external to the Store Driver core into individual agents. These agents are registered to published events. Therefore, the individual agents run during defined events in the message delivery process or in the submission process. This new extensibility model gives the Exchange 2010 Store Driver the following advantages over earlier versions of Exchange:

  • An abstraction layer exists between the Store Driver core and the agents. This provides defined arguments and classes for agents to operate on e-mail messages during processing.

  • Process isolation is improved. The increased modularity of the agent configuration means that if the Store Driver appears to be crashing, you can more easily determine which particular agent causes the crash.

Exchange 2010 includes the following Store Driver agents.

Agent Description Event

Content Aggregation Agent

This agent is responsible for downloading RSS POP3 content based on OWA user-submitted POP3 and RSS subscriptions. Downloaded content is then delivered to the user's mailbox.

OnPromotedMessage

Mailbox Rule Agent

This agent is responsible for retrieving, evaluating and executing the mailbox rules stored in the user’s mailbox.

OnPromotedMessage
OnCreatedMessage
OnDelivered Message

Approval Processing Agent

This agent is part of the Approval Framework and is responsible for intercepting the initiation message during transport, generating an approval request, and updating the original initiation message by using the moderator response after it is received.

OnCreatedMessage

Approval Submit Agent

This agent is also part of the Approval Framework. This agent is responsible for resubmitting approved messages for delivery. It works together with the approval assistant to submit the initiation message or the rejection message to the transport pipeline for delivery.

OnDemotedMessage

Additionally, Store Driver supports the following agents.

Agent Description Event

Mobile Message Receive Agent

This agent is used to resolve the From field from the phone number format of an SMS message to the GAL entry of the recipient.

OnPromotedMessage

Conversation Processing Agent

This agent is responsible for stamping a conversation ID/index on messages so that the messages can be associated with related messages when the messages are viewed by clients such as Outlook or OWA.

OnPromotedMessage

Message Records Management Agent

This agent is used to stamp messages by using retention tags based on retention policies defined for the organization. The MRM assistant then removes data according to the retention tags.

OnCreatedMessage

UM Play-On-Phone Agent

The Play-On-Phone feature enables UM-enabled users to access a voice mail message. However, instead of playing the media file through their computer speakers, users can listen to the message on a telephone. This agent handles the Play-On-Phone request from the mobile device to trigger an outgoing telephone call.

OnPromotedMessage

Meeting Message Processing Agent

This agent is responsible for processing meeting acceptance, cancellation, and alternative proposal requests against mailbox calendars.

OnCreatedMessage

Meeting Forward Notification Agent

This agent is responsible for sending a notification to the meeting originator if the meeting request is forwarded to a different user.

OnDemoted Message

This alert may be triggered when one of the following conditions is true:

  • One of the Store Driver agents is failing

  • The Store Driver cannot communicate with the Microsoft Exchange Mail Submission service

For more information, see Transport Architecture.

User Action

To resolve this problem, do one or more of the following:

  • Review the Application log and System log on your Exchange 2010 servers for related events. For example, events that occur immediately before and after this event may provide more information about the root cause of this error.

  • To review detailed information about the cause of this alert, use the Operations Console in Operations Manager. For more information, see the "Introduction" section in this topic.

  • Examine the event log entries for each MSExchangeStoreDriver event to determine whether an exception occurs. You may have to increase diagnostics logging to generate additional events.

  • Restart the Microsoft Exchange Transport service. Then, examine the event log to verify that the Store Driver loaded successfully. You may have to increase diagnostics logging to log the component startup sequence during the startup process of the Microsoft Exchange Transport service. To increase diagnostics logging for the Transport component, follow these steps:

    1. Start Registry Editor (Regedit.exe), and then locate the following registry subkey:

      HKEY_LOCAL_MACHINE\SYSTEM\Current Control set\Services\MSExchangeTransport\diagnostics

    2. Set the following REG_DWORD values to 7:

    • Smtpreceive

    • Smtpsend

    • DSN

    • Components

    • Remote Delivery

    • Categorizer

  • Use the Exchange Troubleshooting Assistant (ExTRA) to obtain Event Tracing for the Store Driver. The following table contains Store Driver trace tags together with their descriptions.

    • Tag name

    • Description

    StoreDriver

    Tracing the Store Driver operation

    MapiSubmit

    Store Driver mail submission tracing from the transport (Store Driver) to the store

    MapiDeliver

    Store Driver local mail delivery tracing from the Store Driver to the store

    MailSubmissionService

    MailSubmission service tracing and RPC client interface in the MailSubmission component that interacts with the transport

    BridgeheadPicker

    Tracing for the MailSubmission BH Picker component

    CalendarProcessing

    Tracing for the agent that runs the CalendarProcessing in the transport

    ExceptionHandling

    Tracing for the Store Driver exception handling

    MeetingForwardNotification

    Tracing for the MeetingForwardNotification agent (MfnSubmitter is responsible for creating and submitting a meeting forward notification)

    ApprovalAgent

    Tracing for the Approval Agent processing

    MailboxRule

    Tracing for the Inbox Rule Agent processing

    ModeratedTransport

    Tracing for the for the processing of Approval message by the Approval Agent and by the Approval Submitter Agent

    Conversations

    Tracing the conversation tracing agent processing that is done in the transport

    MailSubmissionRedundancyManager

    Tracing the Mail Submission service Shadow Redundancy Manager

    UMPlayonPhoneAgent

    Tracing for the UM Protected Play-On-Phone agent processing

    SmsDeliveryAgent

    Tracing for the SMS delivery agent

    UMPartnerMessageAgent

    Tracing for the Processes UM partner requests and response messages (currently used by the PartnerFax and PartnerVoiceMailPreview features)

    FaultInjection

    For test code

    To configure tracing, follow these steps:

  1. On the HUB Transport server, start the Exchange Troubleshooting Assistant, %ProgramFiles%\Microsoft Exchange Server\Bin\Extra.exe.

  2. After the program starts, check for updates, and then click Next.

  3. In the Welcome screen, click Select a task.

  4. Click the Trace Control option.

  5. Set the max trace size option to 500, and then click set manual trace tags.

  6. Under Trace Types, click to select the following check boxes:

    • PFD

    • Fatal

    • Error

    • Warning

    • Info

    • Debug

    • Function

  7. Under components to trace, click to select the following check boxes:

    • ADProvider

    • Common

    • Common Logging

    • Mapinet

    • Mapiprovider

    • Store

    • Store driver

    • Transport

  8. Create a message that enables you to reproduce the problem, but do not send the message.

  9. Click Start Tracing.

  10. Try to reproduce the problem by sending the affected message or attachment.

  11. When you receive the NDR, verify that it is from the server on which you are running ExTRA.

  12. Resend the affected message to verify that you reproduced the problem. Then, stop tracing.

  13. After you stop tracing, the path to the saved trace file is displayed in the ExTRA tool. By default, the file is named Exchsrvr\Bin\ExchangeDebugTraces.etl. You can use this file to obtain additional support from Microsoft Customer Support Services (CSS).

    For more information, see Another Method of Obtaining ExTRA Traces.

    To obtain the Exchange Troubleshooting Assistant, see Microsoft Exchange Troubleshooting Assistant v1.1.

    The content of each blog and its URL are subject to change without notice. The content within each blog is provided "AS IS" with no warranties, and confers no rights. Use of included script samples or code is subject to the terms specified in the Microsoft Terms of Use.

  • Resolve your issue by using self-support options, assisted support options, and other resources. You can access these resources from the Exchange Server Solutions Center. From this page, click Self-Support Options in the navigation pane to use self-help options. Self-help options include searching the Microsoft Knowledge Base, posting a question at the Exchange Server forums, and other methods. Alternatively, in the navigation pane, you can click Assisted Support Options to contact a Microsoft support professional. Because your organization may have a specific procedure for directly contacting Microsoft Product Support Services, be sure to review your organization's guidelines first.

For More Information

If you are not already doing so, consider running the Exchange tools, which have been created to help you analyze and troubleshoot your Exchange environment. These tools can help make sure that your configuration aligns with Microsoft best practices. They can also help you identify and resolve performance issues and improve mail flow. To run these tools, go to the Toolbox node of the Exchange Management Console. To learn more about these tools, see Managing Tools in the Toolbox.