Messaging (Middleware)---a Technical Reference Guide for Designing Mission-Critical Middleware Solutions

Messaging scenarios are characterized by asynchronous, loosely-coupled communications between message senders and receivers, which are commonly referred to as publishers and subscribers in a publish/subscribe messaging pattern (pub/sub). Messaging solutions attempt to relax tightly coupled communication by introducing one layer of abstraction through an intermediary, which is often referred to as message-oriented middleware.

When communication is performed using an intermediary, message senders do not need to have precise knowledge of their receivers. As such, the relationship between publisher and subscriber can be one-to-one or one-to-many. One-to-one scenarios may also be referred to as point-to-point, and are often satisfied with queue-based messaging. One-to-many scenarios may also be referred to as fan-out, broadcast, or multicast, and are often achieved with pub/sub.

Many providers of enterprise messaging software offer a similar set of features (which are often used by integration platforms) including:

• Durable and non-durable messaging

• Transactional and non-transactional messaging

• Support for distributed transactions and transaction propagation

• Guaranteed delivery

• Scale-out, high availability and failover

• Message correlation

• Management and monitoring tools

• Delivery assurance for exactly once, at least once and/or best effort (may be configurable)

• Ordered delivery

• Permanent and temporary queues and topics

• Poison message management

• Reliable

• Request-response and one-way communication patterns

• Priority Messaging

Best Practices

Development

There is not currently a one-size-fits-all messaging platform. The following section provides high level guidance for choosing the messaging platform that fits your scenario.

• For building a publish/subscribe messaging solution that can be hosted in-process and does not require durability, consider WCF Duplex Services or WCF Peer Channel.

• For building a publish/subscribe messaging solution that needs to traverse geographies or organizations, consider the AppFabric Service Bus and associated .NET Service Bus Bindings.

• For a queue-based messaging solution that can be hosted in-process, support durability, and provides for configurable poison message handling, consider MSMQ and WCF NetMsmq.

• For cloud based queue-centric messaging use Windows Azure Queues.

• For a centralized messaging platform that provides a rich administrative interface, long-running transaction support, and integration with third party systems, including other messaging systems, use BizTalk Server.

• For a centralized messaging platform that provides for asynchronous triggers, large scale batch processing, and other database-centric capabilities, use SQL Service Broker.

• For discussion on various factors related to loose coupling and composition and how they affect agility, refer to https://msdn.microsoft.com/en-us/architecture/aa480028.aspx.

Deployment

• AppFabric (Custom WCF and MSMQ messaging)—https://msdn.microsoft.com/en-us/windowsserver/ee695849.aspx

• MSMQ—https://technet.microsoft.com/en-us/library/cc730960.aspx

• BizTalk

  • https://msdn.microsoft.com/en-us/library/aa561567(v=BTS.70).aspx

  • https://msdn.microsoft.com/en-us/library/aa548040(v=BTS.70).aspx

• SQL Service Broker—https://msdn.microsoft.com/en-us/library/bb522900.aspx

Management and Administration

• AppFabric—https://msdn.microsoft.com/en-us/library/ee677254.aspx

• MSM— https://technet.microsoft.com/en-us/library/cc739189(WS.10).aspx

• BizTalk—https://msdn.microsoft.com/en-us/library/aa560906(v=BTS.70).aspx

• SQL Service Broker—https://msdn.microsoft.com/en-us/library/ms162211.aspx

Case Studies and References

Examples of successful architectures are described in the following case studies and white papers:

• AppFabric—https://www.microsoft.com/casestudies/Case_Study_Detail.aspx?CaseStudyID=4000007145

• BizTalk—https://www.microsoft.com/biztalk/en/us/case-studies.aspx

• SQL Service Broker—http://sqlcat.com/whitepapers/archive/2009/03/24/service-broker-performance-and-scalability-techniques.aspx

Questions and Considerations

This section provides questions and issues to consider when working with your customers.

Development

• Understand quality of service requirements. Do you need durable messaging? Do you need to support store and forward?

• Do you need to be able to host the messaging system in-process?

• Know your platform’s strategy for handling poison or failed messages. Does your system back up when a message cannot be processed or does it simply store it and move on? This is one of the most overlooked aspects of messaging. Do not delay your understanding of this aspect of your messaging architecture until deployment!

• Understand your platform choices for delivery guarantees. Do you need once only, at least once, or best effort? Do you need in-order guarantees? Are your SLAs primarily based on dependability or speed?

• What are the development skills of your team? Are they primarily traditional .NET developers? Integration specialists? DBAs?

• How much interoperability and third party integration is required for the messages your publishers and subscribers will exchange?

• What model are you looking for? Do you require a strongly typed model where everything is decided up front, or are you looking for more dynamic and flexible messaging paradigm?

• Does your messaging system need to support transactions, either local or distributed?

Deployment

• Does your system support HA?

• How does your system need to scale? Horizontally, vertically, or elastically?

• Do you need to centralize the inventory of publishers and subscribers in a shared discoverable repository?

• Do you have publishers and subscribers across business partners, networks, or geographies?

• Do you need to be able to run publishers and subscribers on client OSs or commodity hardware?

Management and Administration

• Do you require a rich graphical management interface?

• What level of security detail does the platform offer?

• What level of automation of maintenance is required?

Performance

• Is latency or throughput more important? What are your targets and Maximum Sustainable Rate requirements?

• Do you need to be able to accommodate multiple latency profiles?

• Do you need to support both durable and non-durable flows to achieve optimal performance?

Appendix

< Full URLS for Hyperlinked Text>