Transactions Across BizTalk Server 2004

Puneet Gupta

Avanade, Inc

May 2005

Applies to: Microsoft BizTalk Server 2004

Summary: Details transactional characteristics of Microsoft BizTalk Server 2004 and how they apply across components, such as messaging, pipeline, and orchestration. Describes how to construct business processes to guarantee message delivery. (25 printed pages)

Microsoft BizTalk Server 2004 is a server application that enables businesses to integrate disparate, heterogeneous applications, and to automate business processes. BizTalk Server 2004 handles all data communication between the underlying Microsoft Windows Server operating system and Microsoft SQL Server database on behalf of all external applications, services, processes, and systems.

To enable the integration of disparate applications and the coordination of logic between business processes, BizTalk Server 2004 transforms and persists all the messages in the MessageBox database on SQL Server. BizTalk Server uses the Publish / Subscribe metaphor for this integration.

The following figure shows the message flow in BizTalk Server 2004.

Figure 1 Message flow in BizTalk Server 2004

Message flow in BizTalk Server 2004
  1. BizTalk Server receives messages from external applications, services, processes, and systems. BizTalk Server uses receive pipelines to convert messages from their external format to XML data and persists these messages to the MessageBox database. This completes the "Publishing" of messages.
  2. BizTalk Server may then process the messages with orchestrations by using the Business Rule Engine with these processes. These processes are considered as "Subscribers" to the messages that were published to the MessageBox database. The processes may have a need to send the message to other applications or to trading partners, which results in the orchestration processes "publishing" a new message to the MessageBox database.
  3. BizTalk Server uses send pipelines to convert the XML data to their external format. The Send adapter then sends the messages to their external applications, services, processes, and systems.

The document explains how the various BizTalk Server adapters handle these transactional semantics.

It will also describe how and when to use atomic and long running transactions associated with business processes.

Adapters in BizTalk Server are the first and last components that typically touch messages received and sent through BizTalk Server. Out of the box, BizTalk provides File, FTP, SQL, HTTP, SOAP, SMTP, Message Queuing, and Base EDI adapters.

This section covers how the transactional semantics apply to messages when they are processed by the following BizTalk adapters. You can use all the adapters to send and receive messages.

Transactions guarantee one-time delivery at a cost. For non-transactional adapters, in a single MessageBox database scenario, the messaging engine uses SQL transactions versus creating COM+ transactions using MSDTC. In these scenarios, however, if there were any system failure while processing, the system could receive or send the same message twice. With transactional adapters (SQL/mySAP/MQ Series/MSMQ), the engine will always use COM+ DTC transactions, since the transactions span multiple systems—the message box(es) and the source or destination of the message. The adapter always owns the transactions and is responsible for the commit or rollback of the transaction; the engine simply votes.

In high throughput scenarios there might be a need to scale out the MessageBox database layer that BizTalk uses. In a multi-message box topology the agent/engine automatically uses DTC and any performance boost achieved by adding the second message box is offset by distributed transactions. Only when a third message box is added does it really improve performance. Hence, the general recommendation is to go from a single message box topology to three message boxes (not to two).

There are special cases when the messaging engine automatically uses a DTC. This is when the engine is processing large files or large batches (batches containing multiple messages whose aggregate datasize is greater than a configured threshold). This is a configured threshold that defaults to 100K.

The following sections cover how the transactional semantics apply to messages when they are processed by the following BizTalk adapters. See BizTalk Server 2004 Help for more information about configuring and using the adapters.

There are several native adapters for BizTalk Server. These include:

  • File adapter
  • SQL adapter
  • Message Queuing adapter
  • FTP adapter
  • HTTP adapter

File Adapter

You use the File receive adapter to read messages from the file system (local or network), and submit them to BizTalk Server. If the network location is unavailable, the adapter retries the operation based on the settings in the receive location (this does not apply to local shares). After the message is accepted successfully, the adapter deletes the file. If the message was read successfully but the failed in the pipeline, the adapter puts the message in the Suspended Queue and deletes the file. If the adapter cannot submit or suspend the message to the MessageBox database, it does not delete the original file.

The receive adapter submits messages to the MessageBox database in batches. The batch size defaults to 20 and is configurable in the receive location properties. If the adapter can successfully read and submit the batch of messages to the MessageBox database, the corresponding files are deleted from the receive location. If some of the messages within the batch failed to be processed, the messages are suspended and the corresponding files deleted from the receive location. If any of the messages fail to be stored to the MessageBox database, the entire batch is rolled back and the corresponding files are left unchanged in the receive location. It is important to note that the batch is only rolled back on an unsuccessful post to the MessageBox database, either in the Suspended Queue or the Processing Queue.

The File send adapter works the opposite way in that, it writes the message to a file and if successful, deletes the message from the MessageBox database. The batch operation applies to the send adapter too, however, the batch size is not configurable and is preset to ten. If for whatever reason, the server cannot write the messages in a batch to files; those messages are rolled back to the MessageBox datbase for retry processing. It is important to note that only the messages that fail to be written are retried, the successful messages in the batch are deleted from the MessageBox database.

The File adapter is a non-transactional adapter, and it is possible to loose files or receive duplicate files.

SQL Adapter

The SQL adapter is classified as a transactional BizTalk Server adapter. The SQL receive adapter is a polling service that periodically polls Microsoft SQL Server for result sets. The receive adapter supports SELECT statements and stored procedures that return single result sets. If a message fails validation or routing in BizTalk Server 2004, the adapter commits the transaction, and the failed message is put in the Suspended queue. The adapter uses MSDTC to manage the transactions.

The SQL send adapter can be configured to send UpdateGrams or to run stored procedures. All UpdateGrams contain the same basic structure:

         <TableName col1='somevalue' col2='somevalue' />
         <TableName col1='somevalue' col2='somevalue' />

The following definitions describe the role of each block:


Identifies the existing state (also referred to as "the before state") of the record instance, and works as the WHERE clause in an SQL statement.


Identifies the new state to which data is to be changed.


Contains the <before> and <after> blocks. A <sync> block can contain more than one set of <before> and <after> blocks. If more than one set of <before> and <after> blocks exist, these need to be specified as pairs (even if they are empty). There can be more than one <sync> block and each <sync> block serves as a unit of transaction. If multiple <sync> blocks are specified in an UpdateGram, the failure of one <sync> block does not affect other <sync> blocks.

The transactional capabilities of the adapter are on by default; there is no way to turn off this capability. Since the transaction spans the application database, the application database design and transactional aspects have an impact on the overall concurrency and performance of the application.

Message Queuing Adapter

The Message Queuing adapter (also known as MSMQ/T) follows the same network protocol as Microsoft Message Queuing, but the persistence is to the BizTalk MessageBox database. The MSMQ/T receive adapter interfaces with receive locations versus queues.

By default, receive queues are transactional. Queues can be specified as non-transactional by setting the Non-Transactional property in the receive location. When using MSMQ Protocol to send a message to MSMQT the scope of the transaction on the client is only to make sure that the message has been successfully written to the queue storage on the local client, and that this message can be safely reconstructed and sent to the recipient even if the client computer were to fail. Also, when using transactions, the message won't be visible on the queue until the transaction is committed.

The destination Microsoft Message Queuing or BizTalk Message Queuing service is responsible for sending acknowledgments to the source Microsoft Message Queuing or BizTalk Message Queuing service, indicating the sequence number of the last received message. If the source Microsoft Message Queuing or BizTalk Message Queuing service does not receive the appropriate order acknowledgment within a certain time limit, it will retransmit the message. Microsoft Message Queuing or BizTalk Message Queuing service drops messages that are received out-of-order by the destination, because the source Microsoft Message Queuing or BizTalk Message Queuing service will resend them.

MSMQ/T always does in-order delivery, which is detrimental to performance.

FTP Adapter

The FTP adapter allows movement of data to and from an FTP server to BizTalk Server. The FTP adapter supports FTP transmissions using default FTP servers available on the following platforms: Solaris 9.0, HP-UX, LINUX (Redhat 7.x), IBM O/S 390 running MVS, AS400 OS/400 V5R1, Microsoft Windows® 2000 Server Service Pack 4, Microsoft Windows 2000 Advanced Server Service Pack 4 and Microsoft Windows Server™ 2003.

The FTP receive adapter guarantees one time delivery of the messages from the Host server through the use of temporary folders. The Temporary folder is set through a property setting for the receive location. The following is a sample scenario:

Server A is the remote FTP server that BizTalk is trying to connect to, and Server B is BizTalk Server with the FTP adapter. The following values need to be setup for the receive location:

Server: Server A

Folder: FTP folder on the remote FTP server

Temporary Folder: C:\FTP\Temp

The FTP folder should exist on Server A with the right permissions. The folder "C:\FTP\TEMP" needs to exist with the right permissions on the BizTalk Server.

The FTP send adapter uses a Temporary folder for error recovery and atomic uploads as well. The error recovery logic uses the size of the file in the Temporary folder (in bytes) to determine the number of bytes already transferred (possibly in the previous attempt). This is only supported for uploads in binary mode.

HTTP Adapter

The HTTP adapter allows the exchange of information between BizTalk Server and an application by means of the HTTP protocol. Applications can send messages to a server by sending HTTP POST or HTTP GET requests to a specified HTTP URL. The HTTP adapter receives the HTTP requests and submits them to BizTalk Server for processing. Similarly, BizTalk Server can transmit messages to remote applications by sending HTTP POST requests to a specified HTTP URL.

The HTTP receive adapter uses a BizTalk-specific Internet Server API (ISAPI) filter to service requests. The HTTP receive adapter submits messages to the server in batches. If some of the messages within a batch fail processing in the pipelines, the HTTP receive adapter returns the server error code "500 - Internal Server Error" to the clients whose messages failed processing, and does not suspend failed messages. If some or all of the messages fail to be stored in the MessageBox database, the system rolls back the entire batch operation and sends server error code 500 to all clients who had requests in the batch. The HTTP send adapter does not support batching operations.

There are other Microsoft adapters and custom adapters in addition to the native adapters. These include:

  • MQSeries adapter 2.0
  • MSMQ adapter
  • mySAP 2.0 adapter
  • Custom adapters

MQ Series Adapter 2.0

The MQ Series adapter provides connectivity with IBM MQ Series. The adapter has two parts: the adapter running on the BizTalk servers and a COM+ component running on an MQ Series Server on the Windows operating system. The adapter supports batching and transactions for both sending and receiving messages. The adapter, however, will need MQ Series Server and cannot work with the MQ Series Client. This is mainly because only the MQ Series Server supports distributed transactions using MSDTC. The adapter guarantees delivery of messages and non-duplication of messages between MQSeries queues and BizTalk Server by starting a transaction that both BizTalk Server and MQSeries enlist in. The MQ Series client version on Windows does not support DTC. Distributed transactions are important to ensure that the message is either in a BizTalk Server MessageBox database or in MQ Series queues, and that the same message is never in both.

The MQ Series receive adapter will retrieve messages in batches from MQ Series. It uses a combination of maximum messages in a batch as well as the batch size properties specified on the receive location. If the receive location is set up to support transactions, the adapter will suspend any of messages that fail validation in the MessageBox database. If the messages cannot be suspended or committed to the MessageBox database, the adapter will rollback the entire batch. In a scenario where some of the messages in the batch are suspended, the adapter resorts to sending messages one at a time to proceed past the error before resorting back to batching. The adapter also supports ordered delivery. The receive location can be set up with two options: Order with Stop and Order with Suspend. The Order with Stop option will end the transaction, causing the message to be rolled back in the MQ Series queue, and the receive location will be disabled. The Order with Suspend option cause the messages to be sent to the Suspend queue, but the receive location will continue to receive other messages. However, the ordered delivery is not maintained in this scenario. See the adapter configuration for restrictions about the Host configuration for ordered delivery.

The MQ Series send adapter also sends messages in batches and follows the same transactional semantics as the receive adapter. If the messages in a batch fail to be written to the MQ Series queue, those messages are suspended, and are handled through retry processing configured for the send port. Similar to the receive adapter, the send adapter will resort to sending messages one at a time in this scenario before recovering back to resume the batch processing. The batch size property applies to the send handler, and is not available on individual send ports, unlike the receive locations.

MSMQ Adapter

The MSMQ adapter, also known as MSMQ/C (MSMQ Classic) adapter, allows sending and receiving messages to MSMQ queues. The adapter has support for both Message Queuing 2.0 and 3.0. The adapter works with transactional and non-transactional, public and private, and local and remote queues. The adapter also provides large message support (greater than 4 MB) through the large message extensions in BizTalk Server.

The receive adapter receives messages in batches. The default is set to 20. The adapter can be configured to do transacted receives. This is limited to local queues only. If the receive location is set up to read transactionally and any of the messages in the batch fail validation or routing in the pipelines, the adapter does the following:

  • Creates a new batch enlisting in the same original transaction and moves the messages to the Suspended queue.
  • Resubmits the original batch.
  • If both of the above actions are successful, the adapter commits the original transaction; otherwise, it rolls back the messages to the queues.
    • The send adapter behaves similarly when configured for transactional sends. The transactional queues can be local or remote. The default batch size for send is five. If the send port is configured to send messages transactionally and any of the messages in the batch fail to be written to the destination queues, the batch is rolled back to the MessageBox database for retry processing.

mySAP 2.0 Adapter

The mySAP 2.0 adapter enables integration with SAP Web Application Server 6.20 and above, and R/3 4.x. The adapter uses the SAP Connector for Microsoft .NET (version 1.0.3). The SAP adapter supports the following SAP schema types:

Business API (BAPI)

Intermediate Document (IDoc) versions 2.0 and 3.0

Remote Function Call (RFC)

By default, the receive adapter receives a single message. You can configure this message by setting a higher value for the batch size. SAP interfaces with program IDs associated with the RFC_Destination configured for SAP. The receive location needs to be configured with the same program ID. If the receive location is disabled or the program ID does not match, SAP will show a failure to transmit the message. If a message fails validation or routing in BizTalk Server 2004, the messages will appear in the Suspended queue. If any of the messages fail to be stored to the MessageBox database, the messages in the batch are rolled back into SAP. The adapter uses MSDTC to manage the transactions.

The send adapter follows the same transactional semantics and if messages cannot be successfully delivered to SAP, messages get suspended and are handled through retry processing configured for the send port. The batch size property applies to the send handler and is not available on individual send ports unlike the receive locations.

Custom Adapters

The receive adapters submit messages to BizTalk and can be developed as a one-way or a two-way request/response pattern. Transacted receivers use a DTC transaction to ensure atomicity between their data source and the BizTalk MessageBox database. A request-response receive adapter will only use a transaction for the submission of the request message; the response message will need to enlist in a separate transaction. The scope of the transaction is between the adapter and the MessageBox database, and the request message will not be published until the transaction for the request message is committed.

The send adapters can also be developed as a one-way or a two-way solicit response pattern. The adapter sends the data in the message to a resource about its enlistment in a DTC transaction. Unlike the two-way receive adapters, the solicit response send adapters can use the same DTC transaction for the solicit response message pair to ensure atomicity for the message exchange.

Some .NET-based components are designed to be used as serviced components and do not allow the transaction to be programmatically committed or aborted, but instead is automatically committed by the platform. For these scenarios, the adapter should use Bring Your Own Transaction (BYOT). This allows the adapter to create a DTC transaction, create the .NET-serviced component that uses the transaction, and allow that component to inherit the created transaction rather than create their own transactions. It is recommended to use the BYOTTransaction helper class that ships with the SDK BaseAdapter. See BizTalk Server 2004 Help for more information about developing custom adapters.

There are scenarios where pipeline components need to enlist in transactions, and either commit the results or roll back depending on whether the messages are successfully posted to the MessageBox database.

The support is for MSDTC transactions and applies to transactional and non-transactional receive adapters. Send adapters are currently not supported.

For a non-transactional adapter, the DTC transaction is initiated by the messaging engine. For transactional adapters, the DTC transaction is initiated by the receive adapter and passed to the engine and then to the component. The transaction applies to the transport batch. Any other component from any receive pipeline that processes the messages from this transport batch can avail of this transaction. If all the messages in the batch are successfully submitted, the transaction is committed by the engine. The messaging engine rolls back the transaction if submission of any of the messages in the batch fail. For non-transactional adapters, if the pipeline component throws an exception, the engine rolls back the batch submission and reports a submission failure to the receive adapter. For a transactional adapter, the engine reports the batch failure to the receive adapter and the receive adapter is responsible for rolling back the batch. A transactional pipeline component has to query the IPipelineContextEx interface and use the GetTransaction method to get the transaction context as detailed in the following example.

The following code example demonstrates a transactional pipeline component that gets the transaction from the EPM and then updates the database within that transaction. This sample assumes that the schema elements that are needed in the database call are Promoted Properties:

public IBaseMessage Execute(IPipelineContext pc, IBaseMessage inMsg)
string customerId;
string companyName;

// Get the customer Id and companyName from the property schema
IBaseMessageContext ctx = inMsg.Context;
customerId = ctx.Read("customerID", 

companyName = ctx.Read("companyName", "http://somepropertyschema")).ToString();

IPipelineContextEx pcx = pc as IPipelineContextEx;

if (pcx != null)
ITransaction trans = null;
trans = (ITransaction)pcx.GetTransaction();

if (trans != null)
SqlConnection nwindConn = 
new SqlConnection(
"Data Source=localhost;Integrated Security=SSPI;" +
"Initial Catalog=Northwind;Enlist=false;");
SqlCommand cmd = 
new SqlCommand(
"INSERT INTO Customers (CustomerID, CompanyName) " +
"Values(@CustomerId, @CompanyName)", nwindConn);
cmd.Parameters.Add("@CustomerId", SqlDbType.VarChar, 5).Value = customerId;
cmd.Parameters.Add("@CompanyName", SqlDbType.VarChar, 
40).Value = companyName;
catch (SqlException e)
throw new ArgumentNullException("IPipelineContextEx:GetTransaction();
return inMsg;

BizTalk Server provides a process execution engine that manages the state, applies business logic, and invokes the supporting applications of complex processes and/or transaction sets. Business processes can be composed as discrete pieces of work using atomic transactions that automatically roll back all changes in case of errors or long running, which can contain nested transactions and use custom exception handling to recover from error scenarios. These transactional semantics are typically managed through the Scope construct in the Orchestration Designer. The long running processes can span days, weeks, and longer time durations. The long running processes typically utilize correlation to correlate messages that are received to the messages that may be sent. The orchestration engine typically dehydrates these instances to conserve system resources and re-hydrates the process upon receipt of these correlated messages. The orchestration engine persists the orchestration state to the MessageBox database at known checkpoints to recover from any application or system exceptions.

The orchestration engine saves the entire state of an orchestration instance at various persistence points to allow rehydration of the orchestration instance. The state includes any .NET-based components that may be used in the orchestration, in addition to messages and variables. The engine stores state at the following persistence points:

  • End of a transactional scope (atomic or long running)
  • At debugging breakpoints
  • At the execution of other orchestrations through the Start Orchestration shape
  • At the Send shape (except in an atomic transaction)
  • When an Orchestration Instance is suspended
  • When the system shutdowns in a controlled manner
  • When the engine determines it wants to dehydrate
  • When an orchestration instance is finished

The engine optimizes the number of persistence points as they are expensive, especially when dealing with large message sizes. As shown in the two orchestration instances below, in the orchestration with the Send Shapes in an atomic scope, the engine determines a single persistence point between the end of the transaction scope and the end of the orchestration. In the other orchestration, there will be two persistence points, one for the first Send shape and the second for the Send shape plus end of the orchestration.

Figure 2 Orchestration persistence

Orchestration persistence

Any .NET-based objects you use in orchestrations, either directly or indirectly, must be marked as serializable unless they are invoked in atomic scopes, or if the objects are stateless and are invoked only through static methods. System.Xml.XmlDocument is a special case and does not need to be marked as serializable regardless of the transaction property for a scope.

How does the special handling for System.Xml.XmlDocument work:

When the user defines a variable X of type T, where T is System.Xml.XmlDocument or a class derived from System.Xml.XmlDocument then the compiler will treat X as a serializable object.

When serializing X, the runtime will keep the following pieces of information: (a) the actual type Tr of the object X is referring to (b) the OuterXml string of the document.

When de-serializing X, the runtime will create an instance of Tr (this assumes a constructor that does not take any parameters) and will call LoadXml providing the instance with the saved OuterXml. X will then be set to point to the newly created instance of Tr.

The orchestration engine uses the concept of correlation to match incoming message to an orchestration instance. The engine may spawn different orchestration instances, each of which deals with instance-specific data. Correlation needs to be used in scenarios where there might be a need to send/receive messages asynchronously, especially in the case of long running processes. The engine uses implicit correlation when dealing with request/response scenarios.

Correlation is achieved by creating correlation types and correlation sets based on the correlation type. A correlation type is a list of properties that can be picked from either the application-specific property schema or the GlobalPropertySchema used by BizTalk. It is important to note that the application-specific properties have to be promoted properties and will not appear in the Correlation Properties dialog box if they are identified as distinguished fields. The following figure shows the Correlation Properties dialog box. As is shown, the available properties by default include all the properties from the GlobalPropertySchema.

Figure 3 - Correlation Properties dialog box

Correlation Properties dialog box

A correlation set is a collection of promoted message property values that you use to route an incoming message to an executing instance. The message is evaluated for the message type and every property in the set in order to correctly route it to the instance.

Sample Scenario:

The following orchestration displays a scenario where a message is:

  • Received by an Activating Receive shape that starts the orchestrations
  • Transforming and sending the message out through a certain transport. The correlation set will be initialized in this shape using some promoted properties that can uniquely identify the message.
  • Receiving a response message back from a transport other than the send transport. The shape is defined as "Following Correlation Sets" that was initialized in the preceding Send shape.

Figure 4 Orchestration scenario

Orchestration scenario

It is important to note that if the same scenario had been constructed using a request-response send port, the engine would create an implicit "Initializing Correlation Sets" for the Send, and "Following Correlation Sets" for the Receive shape.

Compensation can be thought of as a process to do a logical undo of the work that has been successfully committed in response to some error condition. This is handled in orchestrations through the use of compensation handlers. Compensation handlers can only be associated with transactional scopes (long running or atomic). It is important to understand that the compensation handler of a scope is invoked only if the scope completed successfully, but then is required to be undone because a surrounding scope has decided to abort (due to failures that may occur later in the process).

The orchestration engine defines clear semantics about the information that is available to a compensation handler:

  • It gets the state of the running instance exactly as it existed at the point at which the associated scope was successfully completed. This includes the state of messages, variables, objects, and so on.
  • A compensation handler cannot influence the global scope of the executing process. Any modification to the global state is only visible locally.

A compensation handler for a particular scope is always invoked by either the compensation or exception handler of the surrounding scope. The default compensation process for nested transactions has the effect of running the compensation handler for each of the nested transactions in the reverse order of how they were committed.

The orchestration engine provides a Compensate shape for invoking specific compensation handlers. If the Compensate shape targets itself (also known as self compensation). The effect is the same as the default compensation. After the compensation handler completes, the control transfers back to the statement immediately following the Compensate shape taht triggered the original compensation. In a nested scope scenario, if the Compensate shape targets a child compensation handler, and if the child scope does not have a user-specified compensation handler, a default system-provided one is run. A particular scope can be compensated only once. The compiler will prevent more than one path leading to the compensation of a particular scope.

The entire orchestration can have a custom compensation handler. It is set by setting the Compensation property for the orchestration to Custom. This has the effect of an additional Compensation tab next to the default Orchestration tab in the Orchestration Designer. An empty sheet is created with all the Port shapes as they exist in the orchestration being compensated. The custom compensation handler is useful to compensate completed orchestration instances and applies primarily for called orchestrations. The custom compensation does not run for stand alone orchestrations.

Samples for compensation usage are provided with scenarios for long running transactions as they typically are used with long running processes where there is a need to undo logical units of work that have completed due to errors in the process.

BizTalk orchestrations can be designed to run discrete pieces of work, following the classic 'ACID' concept of a transaction. These discrete or atomic units of work, when performed, move the business process from one consistent state to a new, consistent and durable state, and that are isolated from other units of work. This is typically done by using the Scope construct that encapsulates the units of work with the transactional semantics. The entire orchestration can also be defined as an atomic transaction without the use of scopes. The scopes, however, cannot be marked as transactional unless the orchestration itself is marked as a long running or atomic transaction type.

Atomic transactions behave like classic MSDTC transactions, but they are not explicit DTC transactions by default. The following isolation levels are supported by the atomic transactions:

  • Read Committed
    Shared locks are held while the data is being read to avoid dirty reads, but the data can be changed before the end of the transaction, resulting in non-repeatable reads or phantom data.
  • Repeatable Read
    Locks are placed on all data that is used in a query, preventing other users from updating the data. This prevents non-repeatable reads, but phantom rows are still possible.
  • Serializable
    A range lock is placed preventing other users from updating or inserting rows into the database until the transaction is complete.

When the atomic transaction fails, all states are reset as if the orchestration instance never entered the scope. The rule BizTalk has for atomic transactions is that all variables (not just local to the scope) participate in the transaction. All non-serializable variables and messages used in an atomic transaction should be declared local to the scope; otherwise, the compiler will give the "variable….is not marked as serializable" error. All atomic scopes are assumed to be "synchronized" and the orchestration compiler will provide a warning for redundant usage, if indeed the synchronized keyword is used for atomic scopes. The synchronization for the shared data extends from the beginning of the scope until the successful completion of the scope (including the state persistence at the end of the scope) or to the completion of the exception handler (in case of errors). The synchronization domain does not extend to the compensation handler.

Atomic transactions can be associated with timeout values at which point the orchestration will stop the transaction and the instance will be suspended. If an atomic transaction contains a Receive shape, a Send shape, or a Start Orchestration shape, the corresponding actions will not take place until the transaction has committed.

An atomic transaction retries when the user deliberately throws a RetryTransactionException or if a PersistenceException is raised at the time that the atomic transaction tries to commit. The latter could happen if for instance, the atomic transaction was part of a distributed DTC transaction and some other participant in that transaction stopped the transaction. Likewise, if there were database connectivity problems at the time when the transaction was trying to commit, there would also be a PersistenceException raised. If this happens for an atomic scope that has Retry=True, then it will retry up to 21 times. The delay between each retry is two seconds by default (but that is modifiable). After 21 retries are exhausted, if the transaction is still unable to commit, the whole orchestration instance is suspended. It can be manually resumed and it will start over again, with a fresh counter, from the beginning of the offending atomic scope.

Only the RetryTransactionException and PersistenceException can cause an atomic scope to retry. Any other exception raised in an atomic scope cannot cause it to retry, even if the Retry property is set to True. The two-second default delay between each two consecutive retries can be overridden by using a public property on RetryTransactionException (if that is the exception that is being used to cause the retries).

The atomic scopes have restrictions on nesting other transactions that cannot nest other transactions or include Catch - Exception blocks. The following scenarios describe the use of atomic transactions.

Scenario 1 - An Atomic Transaction with COM+ ServicedComponent

The following orchestration shows how to use the RetryTransactionException with atomic transactions. Although exception handlers cannot be directly included for an atomic scope, the scope can include a non-transactional scope that can have an exception handler. The ServicedComponent enlists in the same DTC transaction and any exception raised by the component is caught and re-thrown as RetryTransactionException. (This assumes that the Retry property is set to True for the atomic scope).

Note that the orchestration would have been suspended and the action in the MessageAssignment shape would have been rolled back even if the RetryTransactionException is not thrown. This pattern, however, allows building resilience in the application where the retries occur automatically.

Figure 5 An atomic transaction with COM+ ServicedComponent

An atomic transaction with COM+ ServicedComponent

Scenario 2 - Using Transacted Adapters with Atomic Transactions

The following orchestration shows how to use the atomic transactions with the SQL adapter. The whole orchestration is marked as long running with individual atomic transactions for the two logical pieces of work, Inserting a new Customer and Insert Order details for the customer. If for whatever reason, the Order Insert fails, the Customer Insert should be rolled back. The sample uses the SQL adapter to do the database work. As mentioned earlier, the scope associated with an atomic transaction completes when the message is sent to the MessageBox database. This implies that after the engine is successful in sending the message in the Scope_InsertCustomer and Scope_InsertOrder scopes, each one of the scopes commits. The SQL adapter creates a new transaction for the actual Insert of the Customer or the order. The Ports have a property “Delivery Notification” for validating that the message has been successfully sent via the Sent Port. When the Delivery Notification property is set to “Transmitted”, a receive subscription is placed before the Transactional commit point of the Send Operation. However in case of Atomic Scopes, the receive subscription is placed in the enclosing Parent scope. In the scenario where the InsertOrder SQL transaction fails, a "Nack" will be sent back and the "Scope_InsertOrder" commits. After the Sent Port exhausts the configured retries, a DeliveryFailureException will be raised This exception will be caught by the default exception handler, which will run the default compensation process. This will invoke the compensation handler associated with the Scope_InsertCustomer, causing the undo operation of inserting the customer information.

Nesting the two scopes in a long running scope, and invoking the Compensate shape (targeting the long running transaction) from the exception handler for the long running scope will result in the same behavior as described above. The whole orchestration could not be marked atomic as atomic transactions do not allow nested transactions.

Figure 6 Transacted adapters with atomic transactions

Transacted adapters with atomic transactions

Atomic transactions are usually unsuitable for ensuring transactional semantics for long running work because of all the resources that are consumed in preserving the ACID semantics. Long running transactions assist in the implementation of such scenarios by providing a framework that provides some of the ACID semantics inherently (both atomicity and isolation is lost), but provide a programmable way to achieve the rest.

Long-running transactions are similar to atomic transactions in regard to guaranteeing durability and consistency of state at the end of the transaction.

They are also fundamentally different from atomic transactions because:

  • Exception handlers are permitted on long-running scopes.
  • There is no "auto" rollback of state in case of a transaction abort. You can achieve this programmatically through the exception and compensation handlers.
  • Long running transactions permit nesting of other transactions (long running or atomic).

Long running scopes can be associated with a timeout, which is a logical time within which the long-running work must complete. If the scope does not complete within the specified time, a pre-defined system exception TimeoutException is raised.

You can create long running processes by either marking the entire orchestration as long running or having an outer long running scope nest any other scopes. In the former scenario, a system-provided exception handler runs, whereas the latter allows associating specific exception handlers to the outer scope. The default system-provided exception handler will run the compensation handler for each of the successfully completed nested transactional scopes, if any, in reverse order of their completion. You can achieve the same through self compensating by using the Compensate shape in the exception handler for a long running transaction.

Scenario 1 - Using Long Running Transactions with Timeouts

The following orchestration is a representation of how to associate timeouts with long running transactions. Sometime, you may need to interface with legacy systems that operate in a batch fashion. The scenario shows a purchase order being received and sent to the legacy system. The legacy system processes the purchase order and sends back a purchase order acknowledgement. The send operation initializes a correlation set using the purchase order number and the receive operation follows that correlation set. The receive operation is also in a long running scope with a timeout value. The orchestration engine will dehydrate the orchestration instance waiting for the receive. The correlation will ensure that the same orchestration instance is invoked after the message is received. If the purchase order acknowledgement does not arrive within the time interval specified by the timeout values, a TimeoutException will be thrown.

Figure 7 Long running transactions with timeouts

Long running transactions with timeouts

Scenario 2 - Using Long Running Transactions with Custom Compensation

The following two orchestrations demonstrate how to associate and invoke custom compensations associated with entire orchestrations. The scenario is similar to Scenario 2 for Atomic Transactions; inserting a new customer and inserting the order details for the customer. The process remains the same, if the order insert fails, you should roll back the customer insert. The customer insert could be done by a legacy system, and is therefore, demonstrated in a separate callable orchestration. The called orchestration has the Custom property set for compensation, which provides a separate sheet to do the compensation process. The compensation is to delete the newly inserted customer.

The calling orchestration has a long running scope to do the order insert. This scope is nested within an outer long running scope. The outer scope has an exception handler associated to catch any exceptions. The handler uses the Compensate shape to invoke the custom exception associated with the called orchestration to roll back any changes that might have occurred in the call to the orchestration.

Figure 8 Long running transactions with custom compensation

Long running transactions with custom compensation

Figure 9 Called orchestration (main)

Called orchestration (main)

Figure 10 Called orchestration (compensation)

Called orchestration (compensation)

Technical Contributors

Ruslan Yakushev, Suri Narayanan, Kartik Paramasivam, John Taylor, Gregory Vandewiele