• Article

June 2012

Volume 27 Number 06

Microsoft Azure - Comparing Microsoft Azure Queues and Service Bus Queues

By Ricardo Villalobos | June 2012

One of the benefits of cloud computing is that it allows you to scale your computing resources up or down depending on specific workloads. This capability is extremely helpful in situations such as unexpected traffic bursts or in applications with micro-seasonality trends. To take advantage of this feature, software solutions need to be architected so that the different components are independent and solely responsible for the tasks assigned to them. This loose coupling prevents unnecessary bottlenecks that could potentially slow down the entire solution. Figure 1 shows the basic difference between tightly and loosely coupled architectures.

Tightly and loosely coupled architectures and compute resource boundaries
Figure 1 Tightly and loosely coupled architectures and compute resource boundaries

One common way to decouple solutions is to use messaging queues, recognized as an enterprise architecture pattern that facilitates the integration of disconnected components. In the cloud, messaging queues have become the canonical example of how to properly segregate functional layers that perform their job as effectively as possible.

Currently, the messaging architecture pattern can be implemented in the Azure platform in two ways: Azure queues and Service Bus queues. In this article I cover their main differences from a practical point of view, considering aspects such as performance, capacity and security during the basic communication cycle, which involves creating a queue, sending messages and processing the messages.

Performance

The most significant difference between Azure queues and Service Bus queues is their average latency: 10 milliseconds for the former and 100 milliseconds for the latter, assuming that the request is generated from a hosted service located in the same location as the storage account. Both types support up to 2,000 messages per second. From a developer’s point of view, this means that when traffic grows, the number of Service Bus queues has to increase more rapidly than when using Azure queues. Programmatically creating a queue is similar in both frameworks, as shown in the code samples in Figure 2 and Figure 3.

Figure 2 Creating a Service Bus queue using the Azure SDK for .NET

using Microsoft.ServiceBus;
using Microsoft.ServiceBus.Messaging;
string issuer = "<obtained from portal>";
string key = "<obtained from portal>";
TokenProvider tP = 
  TokenProvider.CreateSharedSecretTokenProvider(issuer, key);
// Retrieve URI for the service namespace (created via the portal)
Uri uri = ServiceBusEnvironment.CreateServiceUri("sb", 
          "{service namespace name}", string.Empty);
// Create NamespaceManager for the created service namespace
NamespaceManager namespaceManager = 
  new NamespaceManager(uri, tP);
// Create a new Queue named "NewQueue"
namespaceManager.CreateQueue("NewQueue");

Figure 3 Creating an Azure queue using the Azure SDK for .NET

using Microsoft.WindowsAzure.StorageClient 
  
string accountName = “<obtained from portal>”; 
string accountKey = “<obtained from portal>”; 
  
//Create service client for credentialed access to the Queue service. 
CloudQueueClient queueClient = new CloudQueueClient(queueEndpoint, 
        new StorageCredentialsAccountAndKey(accountName, accountKey)); 
  
//Get a reference to a queue in this storage account. 
CloudQueue queue = queueClient.GetQueueReference("NewQueue"); 
  
//Check whether the queue exists, and create it if it does not. 
if (!queue.Exists()) 
{ 
         queue.Create(); 
}

Capacity

Messaging pattern capacity is limited/defined by three factors: message size, storage supported per queue and the amount of time that messages are kept alive (or TTL, for time-to-live). When it comes to Azure queues, each message can be up to 64 KB, can hold up to 1 TB of data and can store messages for up to 7 days. Service Bus queues support messages up to 256 KB and are limited to 5 GB of data, but their TTL for messages is unrestricted.

These limits are important when it comes to processing messages in both frameworks, particularly when dealing with situations in which the application crashes after processing the message but before the message is removed from the queue. The code in Figure 4 and Figure 5 shows how messages can be sent and received using Service Bus queues (assuming that the queues are already created).

Figure 4 Sending messages to a Service Bus queue

MessagingFactory factory = 
  MessagingFactory.Create(uri, tP); 
MessageSender newQueue = 
  factory.CreateMessageSender("NewQueue"); 
  
// Create message, passing a string message for the body 
BrokeredMessage brokeredMessage = 
  new BrokeredMessage("New Message in queue"); 
  
// Set properties on new message 
message.Properties["Value1"] = "Some Value"; 
message.Properties["Sequence"] = 1;    
// Send message to the queue 
newQueue.Send(message);

Figure 5 Receiving messages from a Service Bus queue

MessagingFactory factory = 
  MessagingFactory.Create(uri, tP); 
MessageReceiver newQueue = 
  factory.CreateMessageReceiver("NewQueue"); 
 while (true)
 { 
    BrokeredMessage receivedMessage = newQueue.Receive();
    if (receivedMessage != null)
    {
       try
       {
          Console.WriteLine(receivedMessage.GetBody<string>());
          Console.WriteLine(receivedMessage.MessageId);
          Console.WriteLine(receivedMessage.Properties["Value1"]);
          // Remove message from queue
          receivedMessage.Complete();
       }
       catch (Exception)
       {
          // Something happened or the process crashed
          message.Abandon();
       }
    }
 }

The code in Figure 6 and Figure 7 shows how messages are sent and received in an Azure queue.

Figure 6 Sending a message to an Azure queue

//Get a reference to the queue previously created 
CloudQueue newQueue = 
  queueClient.GetQueueReference("NewQueue"); 
  
//Remove any existing messages from the queue 
newQueue.Clear(); 
  
//Create a new message 
CloudQueueMessage newMessage = 
  new CloudQueueMessage("New Message in queue"); 
             
//Add the messages to the queue with a TTL of 10 minutes. 
newQueue.AddMessage(newMessage, new TimeSpan(0, 10, 0));

Figure 7 Receiving a message from an Azure queue

//Get message from the previously created queue 
CloudQueueMessage receivedMesssage = newQueue.GetMessage(); 
Console.WriteLine(receivedMessage.AsString); 
  
//Remove message from queue 
newQueue.DeleteMessage(msgRead);

Security

Authentication for storage queues is limited to the use of a security token, which is shared by other components (including tables and binary large objects, or BLOBs). For this reason, the application needs to handle all the security when a more sophisticated mechanism is required. Getting access to the Azure queue is as simple as providing the storage account name and the token associated with it (extracted from the code in Figure2):

string accountName = “<obtained from portal>”; 
string accountKey = “<obtained from portal>”; 
  
//Create service client for credentialed access to the Queue service. 
CloudQueueClient queueClient = 
        new CloudQueueClient(queueEndpoint, 
        new StorageCredentialsAccountAndKey(accountName, accountKey));

For Service Bus queues, Azure Access Control Service (ACS) is used as the primary authentication mechanism, supporting multiple predefined identity providers as well as custom ones. The following code shows how to access a Service Bus queue that is protected using tokens (extracted from the code in Figure 3):

string issuer = "<obtained from portal>";
string key = "<obtained from portal>";
TokenProvider tP = 
  TokenProvider.CreateSharedSecretTokenProvider(issuer, key);
// Retrieve URI for the service namespace (created via the portal)
Uri uri = ServiceBusEnvironment.CreateServiceUri("sb", 
          "{service namespace name}", string.Empty);
// Create NamespaceManager for the created service namespace
NamespaceManager namespaceManager = 
  new NamespaceManager(uri, tP);

In this example, the token provider is called as new tokens are required—including scenarios in which they have expired (the default is 1200 seconds). Predefined and custom providers are configured through the Azure portal and involve presenting the identity provider UI to the user in order to generate a token through ACS. After this process is completed, the application accessing the queue can extract it and provide it for authentication.

Wrapping Up

Because they offer support for sessions, transactions and a pub/sub model, Service Bus queues are the preferred choice when it comes to more complex messaging scenarios. However, Azure queues are a great option for supporting architectures that require high speed and high volume throughput for basic decoupling of their different components.

From a developer’s point of view, the biggest differences between Service Bus queues and Azure queues are the use of different namespaces and the way duplicated or poisoned messages are dealt with. Service Bus queues also offer a more flexible security mechanism through the use of ACS. Advanced scenarios, such as advanced filtering, deferred messages and dead-letter queues, will be covered in future articles.

Ricardo Villalobos  is a seasoned software architect with more than 15 years of experience designing and creating applications for companies in the supply chain management industry. He has numerous technical certifications as well as an MBA in supply chain management from the University of Dallas. He works as an Azure architect evangelist for Microsoft.

Thanks to the following technical expert for reviewing this article:  Bruno Terkaly