Concurrency Issues with the BizTalk Adapters

Article
11/17/2011

This topic discusses several concurrency issues of which you should realize when you are working with the BizTalk adapters that are provided with Commerce Server 2009 R2. These adapters are known informally as the Commerce Server 2009 R2 adapters.

Avoid Infinite Looping When Using the BizTalk Adapters

When you use the Commerce Server 2009 R2 adapters to incrementally move data into and out of the systems for which the adapters are provided, a particular item in one of those systems could be continually misinterpreted as a new item. This causes the item to be imported and exported repeatedly. Each of these systems has provided dedicated properties and techniques for their use to prevent this.

Orders System

The Orders receive adapter uses the two transport properties, Status From and Status To, to make sure that new orders are only exported (for fulfillment) one time. If you configure a particular Orders receive adapter endpoint to have its Status From property set to "New" and its Status To property set to "Submitted," this controls two behaviors when the receive adapter performs an export operation for that endpoint. First, the setting of the Status From property causes the receive adapter to retrieve only new orders from the Orders System. And second, the receive adapter updates the status of that order in the Orders System to "Submitted." Therefore, when the receive adapter runs for this endpoint the next time, again retrieving only new orders, any orders retrieved during the last execution of the receive adapter are not retrieved again.

For more information about how these two properties are used together, see Managing Order Status with the Orders Receive Adapter.

Catalog and Inventory Systems

The Catalog System and the Inventory System use the same mechanism to help make sure that the same items are not repeatedly imported and exported. All the items in these systems have a flag called Export Ready, for which the default value is True unless otherwise specified. By default, when the Catalog and Inventory send adapters import items, they set the Export Ready flag to False. This indicates that this item should not be exported because it was just imported. Subsequently, when a Catalog or Inventory receive adapter endpoint runs in incremental mode, it will only include in the export those items that have changed since the last time that adapter endpoint was run and which have their Export Ready flag set to True. This will include items that have been created or otherwise modified in a way other than by being imported by a send adapter.

Profiles System

The Profiles System uses a date/time property that is associated with every item that it maintains. This property is dedicated to tracking the last time that the item was changed by the Profiles send adapter. Whenever the Profiles send adapter changes an item in the Profiles System, it sets both of the properties CS_Adapter_Date_Last_Changed and Date_Last_Changed to the same date/time value. And whenever the Profiles receive adapter is performing an incremental export, it only includes items for which the Date_Last_Changed property is set to a date/time value that is more recent than its setting for the CS_Adapter_Data_Last_Changed property. This indicates that the item has been changed through some mechanism (such as the Profiles Manager) other than by a Profiles send adapter.

Possible to Miss Catalog or Inventory Items Because of Embedded Last Modified Dates

If a catalog or inventory import operation contains LastModified data, recording the date and time when individual catalog or inventory items were last modified on the system of origin, these date/time entries will be recorded in the Catalog System or Inventory System, respectively, as the LastModified date/time for the imported catalog or inventory items. If a Catalog or Inventory receive adapter endpoint is scheduled to perform incremental exports, it will export any items with a LastModified date/time that is later than the date/time at which that Catalog or Inventory receive adapter endpoint last ran. But because the LastModified date/time of some items might reflect the date/time that they were modified on their system of origin, as recorded in the import message, it might be earlier than when the Catalog or Inventory receive adapter endpoint last ran.

For example, consider the following sequence of events for a Catalog receive adapter endpoint:

A catalog item X on system A is updated at 10:00 A.M.
A Catalog receive adapter endpoint on system B performs a scheduled incremental export to system C at 10:30 A.M.
Catalog item X is included in an incremental export from system A to system B at 12:00 P.M. The export includes its 10:00 A.M. LastModified value.
A Catalog send adapter endpoint for system B imports catalog item X into Commerce Server 2009 R2 at 12:05 P.M. that includes its 10:00 A.M. LastModified value.
The Catalog receive adapter endpoint on system B performs its next scheduled incremental export to system C at 1:30 P.M. This incremental export does not include catalog item X because the receive adapter is only including catalog items modified since 10:30 A.M., the last time that it ran, and catalog item X is recorded as having last been modified at 10:00 A.M., 30 minutes earlier.

There are several possible ways to prevent this scenario that include the following:

You could configure the Catalog receive adapter endpoint on system B to perform an expression-based export operation instead of an incremental export operation. Set the value of the Products And Categories Clause transport property for this receive adapter endpoint to "ExportReady = 1". Also, make sure that you set the Set Export Ready transport property of the Catalog send adapter endpoint in step 4 to True. By setting these two properties as specified here, you will enable these two adapter endpoints to work together without missing any catalog items. The one, potentially fatal, caveat to this potential solution is that only changes imported by the send adapter will be exported by the receive adapter.
Another solution is to make sure that LastModified data is either not included in the export operation in step 3 or removed from the catalog message before step 4. This removal could be performed somewhere in BizTalk Server, such as an orchestration or pipeline. Remember that there are performance implications to consider when messages are large.

Catalog and Inventory Systems Do Not Allow Concurrent Import Operations

If a second Catalog or Inventory import message (CommerceServerCatalogImport or CommerceServerInventoryImport) arrives at a Catalog or Inventory send adapter endpoint when a prior message of the same type is still being processed by Commerce Server 2009 R2, the second such message will be suspended and an exception event will be logged with the text "This operation cannot be performed because an import operation is currently in progress."

Note

A Catalog import message can be processed at the same time as an Inventory import message. This concurrency constraint concerns two or more concurrent Catalog import messages or two or more concurrent Inventory import messages.

Although the Catalog and Inventory Systems do technically allow for concurrent import and export operations, doing this is not advised because the results will not be deterministic.

One solution for preventing concurrent import attempts, at least when the import messages are being processed by the same Catalog or Inventory send adapter endpoint, is to configure the corresponding send port to do ordered delivery of messages. This option can be set on the Advanced tab of the Send Port Properties dialog box.

If you decide on this approach, you should consider establishing different Catalog or Inventory send adapter endpoints to process different types of messages. Otherwise, if you use a single Catalog or Inventory send adapter endpoint to process different types of messages by setting its Endpoint Message Type transport property to From Context, you may have the quick query messages queuing up behind much slower import messages.

Receive Adapters Do Not Support Pooled BizTalk Scenarios

You can configure the Commerce Server 2009 R2 receive adapters to periodically retrieve data from Commerce Server 2009 R2. This includes the ability to retrieve only data that is new since the last time that it was retrieved (retrieving new orders every hour is a good example). The receive adapters store the time that they last ran in the system registry of the BizTalk server associated with the corresponding BizTalk host instance. This architecture is incompatible with the BizTalk feature that allows a host instance to be distributed across a pool of BizTalk servers. If you try to use these features together, the receive adapters' records of when they last ran may be unintentionally recorded and retrieved from the system registries of multiple BizTalk servers. This causes the adapters to construct queries that can result in new data being retrieved more than one time. Therefore, do not use the BizTalk pooling functionality together with the Commerce Server 2009 R2 receive adapters.