Export (0) Print
Expand All

Performance Factors

This topic describes the performance factors for the most commonly used components of a BizTalk Server solution. Some components are used by most BizTalk solutions while others are optional. An understanding of these factors will assist with the decision making process to maximize the performance of a production BizTalk Server environment.

While BizTalk Server imposes no restriction on message size, practical limits and dependencies might require you to minimize the size of your messages because large messages require more processing resources. As message size increases, overall throughput (messages processed per second) decreases. When designing your scenario and planning for capacity, consider the average message size, message type, and number of messages BizTalk Server processes. Do not use unnecessarily long attribute and tag names; if possible, keep the length under 50 characters. For example, do not use a 200-character tag name for a message size of only 1 byte.

If the in-memory size of a received message exceeds the number of bytes specified for the Large message fragment size that is configurable on the Group Properties page for the BizTalk Group in the BizTalk Server Administration console, then the message is split into fragments of the specified size and the fragments are written into the MessageBox under the context of a Microsoft Distributed Transaction Coordinator (MSDTC) transaction as follows:

  1. If the incoming message is being published under the context of an existing MSDTC transaction, then this transaction is used when writing the message fragments to the MessageBox. For example, if the incoming message is being published by a transactional adapter configured to require transactions then the existing transaction would be used when writing the message fragments to the MessageBox.

  2. If the incoming message is not being published under the context of an existing MSDTC transaction, then a new MSDTC transaction is created to write the message fragments to the MessageBox database. In this scenario, the following considerations apply:

    • Increase the value for Large message fragment size to reduce the frequency with which large messages are fragmented and reduce the incidence of creating the associated MSDTC transactions. This should be done because excessive use of MSDTC transactions is expensive from a performance standpoint. Note that increasing this value may also increase the amount of available memory that is used.

    • If it takes longer than the maximum allowable MSDTC transaction timeout of 60 minutes to write a message to the MessageBox, then the transaction times out, an error occurs, and the attempt to write the message fails and is rolled back. The Large message fragment size value should be increased enough to avoid this problem when processing very large messages. Depending on available memory, this value should be increased up to a maximum value of 1000000 bytes.

    • Each message fragment in a message creates one or more SQL Server database locks against the MessageBox database. When the number of locks exceeds several hundred thousand, it is possible that SQL Server will generate “out of lock” errors. If this problem occurs, increase the value for Large message fragment size to reduce the number of fragments (which decreases the number of SQL Server database locks made against the MessageBox database) or consider housing your MessageBox database on a 64-bit version of SQL Server. The number of available locks is significantly higher on the 64-bit version of SQL Server than on a 32-bit version of SQL Server. The following formula can be used to estimate the maximum number of messages per interchange when the MessageBox database is housed on a 32-bit version of SQL Server:

200,000 / (Number of CPUs * BatchSize * MessagingThreadPoolSize)

For more information on how BizTalk Server processes large messages, see “How BizTalk Server Processes Large Messages” in the BizTalk Server 2006 R2 help at http://go.microsoft.com/fwlink/?LinkID=102620.

The throughput for message parsing (especially flat-file parsing) is affected by the complexity of the schemas. As schema complexity increases, overall performance decreases. When designing schemas, reduce the length of node names and move promoted properties to the top of the schema to reduce retrieval time to increase performance.

Depending on the complexity of the maps, map transformation can be resource intensive. As map complexity increases, overall performance decreases. To improve overall performance, minimize the number of links and functoids used in your maps, especially functoids that call external resources such as the DB Lookup functoid.

Because pipeline components have a significant impact on performance (for example, a pass-through pipeline component performs up to 30 percent better than an XML assembler/disassembler pipeline component), make sure that any custom pipeline components perform optimally before implementing them in your deployment. Minimize the number of pipeline components in your custom pipelines if you want to maximize the overall performance of your BizTalk application.

You also can improve overall performance by reducing the message persistence frequency in your pipeline component and by coding your component to minimizing redundancy. Every custom assembly and in particular artifacts that could potentially disrupt performance, like custom tracking components, should be tested separately under heavy load condition to observe their behavior when the system is working at full capacity and to find any possible bottlenecks.

If you need to read the inbound message inside a pipeline component, avoid loading the entire document into memory using an XmlDocument object. The amount of space required by an instance of the XmlDocument class to load and create an in-memory representation of a XML document is up to 10 times the actual message size. In order to read a message, you should use an XmlTextReader object along with an instance of the following classes:

  • VirtualStream (Microsoft.BizTalk.Streaming.dll) - The code for this class is located in two locations under the Pipelines SDK as follows: SDK\Samples\Pipelines\ArbitraryXPathPropertyHandler and SDK\Samples\Pipelines\SchemaResolverComponent\SchemaResolverFlatFileDasm.

  • ReadOnlySeekableStream (Microsoft.BizTalk.Streaming.dll).

  • SeekAbleReadOnlyStream - The source code for this class is located in two locations under the Pipelines SDK as follows: SDK\Samples\Pipelines\ArbitraryXPathPropertyHandler and SDK\Samples\Pipelines\SchemaResolverComponent\SchemaResolverFlatFileDasm.

If XLANG/s is generating an XML message, then we recommend not using the XMLTransmit pipeline on the send side. Doing so incurs processing overhead. We recommended that you use the PassThru pipeline on the send side in all cases except where additional pipeline processing is required.

Use the PassThruReceive and the PassThruTransmit standard pipelines whenever possible. They do not contain any pipeline component and do not perform any processing of the message. For this reason, they ensure maximum performance in receiving or sending messages. You can use a PassThruReceive pipeline on a receive location if you need to publish a binary document to the BizTalk MessageBox and a PassThruTransmit pipeline on a send port if you need to send out a binary message. You can also use the PassThruTransmit pipeline on a physical send port bound to an orchestration if the message has been formatted and is ready to be transmitted. You will need to use a different approach if you need to accomplish one of the following actions:

  • Promoting properties on the context of the inbound XML or Flat File message

  • Applying a map inside the receive location

  • Applying a map in a orchestration subscribing the message

  • Applying a map on a send port subscribing the message

To accomplish one of these actions, you must probe and discover the document type inside the receive pipeline and assign its value (namespace#root-name) to the MessageType context property. This operation is typically accomplished by a disassembler component as the Xml Disassembler component (XmlDasmComp) or the Flat File disassembler component (FFDasmComp). In this case, you need to use a standard (e.g. XmlReceive pipeline) or a custom pipeline containing a standard or a custom disassembler component.

Inside a receive pipeline, you should promote items to the message context only if you need them for message routing (Orchestrations, Send Ports) or demotion of message context properties (Send Ports). If you need to flow metadata with a message, and you don't use them for routing or demotion purposes, use the IBaseMessageContext.Write method instead of the IBaseMessageContext.Promote method.

Acquire resources as late as possible and release them as early as possible. For example, if you need to access data on a database, open the connection as late as possible and close it as soon as possible. Use the C# using statement to implicitly release disposable objects or the finally block of a try-catch-finally statement to explicitly dispose your objects. Instrument your source code to make to make your components simple to debug.

If you need to extract information from a message using an XPath expression, avoid loading the entire document into memory using an XmlDocument object just to use the SelectNodes or SelectSingleNode methods. Instead, you should:

  • Load the message using an instance of the SeekAbleReadOnlyStream class.

  • Create an instance of the XPathReader .NET class that you can find in the Microsoft.BizTalk.XPathReader.dll assembly.

  • Use the methods (ReadUntilMatch, Match, etc.) of the XPathReader instance to find out matches.

The amount of message tracking can have a significant impact on performance. As the number of items tracked and the amount of tracking data increases, overall performance decreases. We recommend putting the Tracking database on a separate server. It is important that you ensure the SQL agent jobs are running. In addition, run the tracking service (TDDS) to move tracking data from the MessageBox database to the Tracking database (BizTalkDTADb) and monitor the Tracking database for disk growth. Archive and clean up the Tracking database regularly by backing up and deleting old files.

BizTalk Server commits all write operations to the MessageBox database, which stores messages and state information for particular message instances. This persistence allows BizTalk Server to ensure data integrity, security, and reliability. As message-persistence frequency increases, overall performance decreases. Whenever possible, reduce the frequency at which data needs to be persisted to the message box. For example, group multiple message-persistence points into one scope.

While scenarios define the particular transport adapters that are required, transport adapters have different settings that you can modify to improve performance. For example, orchestration and adapter functionality should be separated into different BizTalk Server hosts to minimize resource contention. For more information, refer to the article BizTalk Server 2006: Comparative Adapter Study at http://go.microsoft.com/fwlink/?LinkId=116888 and Configuration Parameters that Affect Adapter Performance at http://go.microsoft.com/fwlink/?LinkId=108785.

BAM disk usage configuration

BAM incurs significant overhead when a BizTalk system is under load due to the significant amount of data that is persisted to the BAM database. Therefore, judicious use of disk I/O techniques for the BAM database critically important.

BAM EventStream APIs

Four types of EventStreams are available for use in a BizTalk BAM scenario:

  • DirectEventStream (DES)

  • BufferedEventStream (BES)

  • OrchestrationEventStream (OES)

  • MessageEventStream (MES)

You should choose one of these APIs based on the following factors:

  • If your concern is latency, choose DES, where data is persisted synchronously to the BAM Primary Import database.

  • If your concern is performance and throughput of event insertion, choose an asynchronous API (BES, OES or MES).

  • If you are writing an application that runs on a computer that does not have BizTalk Server installed, use DES and BES, these APIs can be used in non-BizTalk applications.

    Cc615058.note(en-US,BTS.10).gifNote
    There are scenarios in which you may want to mix EventStream types. For example, for pipeline processing, you may want to capture the particular data in BAM regardless of whether the pipeline is rolling back its transaction. In particular, you may want capture data about how many messages failed or how many retries occurred during pipeline processing. To capture the data in this situation you should use BES.

  • If your application runs on a computer on which BizTalk Server is installed, use MES and OES. (These APIs are available only from BizTalk applications.)

    Cc615058.note(en-US,BTS.10).gifNote
    OES is the equivalent of MES but for BizTalk orchestrations.

  • If you want BAM event persistence to be in sync with pipeline transaction, you should use a Messaging Event Stream (MES).

All the asynchronous EventStreams (BES, MES, and OES) persist data first to the BizTalk MessageBox database. Periodically the data is processed and persisted to the BAM Primary Import database by the Tracking Data Decode Service (TDDS).

For more information about the BAM EventStream APIs, see “EventStream Classes” in the BizTalk Server 2006 R2 documentation at http://go.microsoft.com/fwlink/?LinkId=115092.

BAM performance counters

For a detailed list of the performance counters for BAM, see “BAM Performance Counters” in the BizTalk Server 2006 documentation at http://go.microsoft.com/fwlink/?LinkId=115093.

Host configuration has a significant impact on performance. BizTalk Server supports the isolation of individual components (including orchestrations, adapters, pipelines, and tracking) into separate hosts, which are logical entities with specific functionality. Careful consideration should be given to the host design of a BizTalk Server solution. How a project decides to allocate processing to hosts and host instances can be a determining factor for that solution’s performance capability. Below are some key heuristics to consider during creation of host design.

Separation of sending, receiving, processing, and tracking functionality

Configure the deployment topology such that different functionality runs in dedicated host instances. This way each host instance gets its own set of resources (on a 32-bit system, 2GB virtual memory address space, handles, threads). If the server is powerful enough (sufficient CPU headroom, memory) to host multiple host instances, they can all be configured to run on the same physical machine. If not, this also makes it easy to scale-out by moving the host instances to dedicated servers. Running the same functionality on multiple servers also provides a highly available configuration.

The tracking host instance is responsible for moving both BAM and HAT data from the MessageBox database (TrackingData table) to the BizTalkDTADb tracking database and/or the BAMPrimaryImport database. If multiple MessageBox databases are configured the tracking host instance uses four threads per MessageBox. When creating a host, you must consider the “enable tracking” check box. This box should only be checked on hosts that are dedicated to tracking and which will have no other BizTalk artifacts running in that host. Turning off Global tracking significantly reduces tracking overhead, but keep in mind that you must still have at least one tracking host instance and it must be started. Otherwise you can mitigate the need for a tracking host if Global Tracking is turned off and if only the direct event stream from BAM is used.

For more information about creating scalable host configurations, see Optimizing BizTalk Server Performance.

The following performance factors should be considered when implementing orchestrations in a BizTalk Server solution:

Serializing, deserializing, or loading large messages into memory

Avoid loading entire documents into orchestrations using an XmlDocument object. If you need to read a message inside an orchestration, avoid loading the entire document into memory using an XmlDocument object, use XmlReader instead when possible (as described below in the section “Use of appropriate .NET classes in your orchestrations”

Use of promoted properties to access message tags or attributes from an orchestration

If you do need to promote properties, promote only those properties that are used for message routing, filters, and message correlation. The promotion of each property requires the disassembler component (XML, Flat, custom) to recognize the document type and to retrieve data from the message using the XPath expression from the relative annotation contained in the XSD which defines the document type. In addition, each property promotion causes a separate call of the bts_InsertProperty stored procedure when the Message Agent publishes the message to the MessageBox database. If an orchestration needs to access a particular element or attribute contained by an XML document, use one of the following techniques:

  1. Use distinguished fields.

  2. Use the XPath built-in function provided by the orchestration runtime.

  3. if messages are quite small (a few kilobytes) and XML-formatted, you can deserialize the message into a .NET class instance and work with public fields and properties. If the message needs a complex elaboration (custom code, Business Rule Engine policies, etc.) accessing data using the properties exposed by an instance of a .NET class is much faster using XPath expressions. When the business logic invoked by the orchestration has completed, the entity object can be serialized back into a BizTalk message. You can create .NET classes from an XML schema using one of the following tools: XSD tool (.NET Framework 2.0) or SVCUTIL (.NET Framework 3.0).

Performance impact of orchestration persistence points

Persistence of the state of each orchestration and how often this occurs will directly affect the performance of orchestrations in your solution. At design time, ensure that you completely understand the concept of orchestration persistence points. For more information about the how orchestration persistence points work in BizTalk Server, see “Persistence and the Orchestration Engine” in the BizTalk Server 2006 R2 documentation at http://go.microsoft.com/fwlink/?LinkID=108780.

Effects of orchestration dehydration

When many service instances are running at the same time, memory and performance are potential issues and dehydration will be an issue. For more information about orchestration dehydration and rehydration, see “Orchestration Dehydration and Rehydration” in the BizTalk Server 2006 R2 documentation at http://go.microsoft.com/fwlink/?LinkID=106789.

Performance impact of Orchestration complexity

The complexity of orchestrations has a significant impact on performance. As orchestration complexity increases, overall performance decreases. Orchestrations can be used in an almost infinite variety of scenarios, and each scenario might involve orchestrations of varying complexity. Avoid complex orchestrations when possible in favor of a modular approach. In other words, split your business logic into multiple, reusable orchestrations.

If you do need to implement a complex orchestration, define messages and variables into inner scopes whenever possible rather than at the root level. This technique maintains a smaller footprint in memory for each orchestration because variables and messages are disposed of when the flow leaves the scope where the variables and messages were defined. This approach is particularly beneficial when orchestrations are saved to the MessageBox at persistence points.

Performance implications of using logical ports bound to physical ports

You can increase performance by eliminating logical ports bound to physical ports that use the following adapters:

  • SQL Server, Oracle

  • SOAP, WSE, HTTP, WCF

  • MSMQ, MQ Series

In BizTalk Server 2006, send and receive pipelines can be directly invoked from an orchestration using the XLANGPipelineManager class contained in the Microsoft.XLANGs.Pipeline.dll. Thus, the processing of pipelines can be moved from ports to orchestrations, while logical ports in an orchestration can be substituted with an Expression Shape which invokes an instance of a given .NET class (for example, a SOAP proxy classes or a Data Access component using ADO.NET). Before adopting this technique, you should be aware that if you do not use adapters and physical ports, you lose the ability to leverage their functions, such as batching, retries, declarative configuration and secondary transports.

Use of appropriate .NET classes in your orchestrations

In general, we can divide the .NET classes used inside an orchestration in two separate categories:

  • Helpers and services - These classes provide common services to orchestrastions such as tracing, error handling, caching, and serialization/deserialization. Most of these classes can be implemented as static classes with no internal state and multiple public static methods. This approach avoids creating multiple objects of the same class in different orchestrations running at the same time, which helps to reduce the working space of host processes and save memory. A class that is stateless helps to reduce the overall size of the internal state that must be serialized and persisted to the BizTalk MessageBox when an orchestration is dehydrated.

  • Entities and Business Objects - You can use these classes to manage entities, such as orders, order items, and customers. A single orchestration can internally create and manage multiple instances of the same type. These classes are typically stateful and expose public fields and/or properties along with methods to modify the internal state of the object. Instances of these classes can be dynamically created by deserializing an XLANGMessage part into a .NET object using the XmlSerializer or the DataContractSerializer classes or by using the XLANGPart.RetrieveAs method. You should structure an orchestration using non-transactional scopes in such a way that instances of stateful classes are created as late as possible and released as soon as they are no longer needed. This approach reduces the working space of host processes and minimizes the overall size of the internal state that is serialized and persisted to the MessageBox database when an orchestration is dehydrated. For more information about using orchestrations in Microsoft BizTalk Server 2004 and BizTalk Server 2006, see the article “FAQ for BizTalk Server Orchestrations” at http://go.microsoft.com/fwlink/?LinkID=116886.

Use of the Call Orchestration shape versus the Start Orchestration shape

Avoid the Start Orchestration shape and use the Call Orchestration shape to execute a nested orchestration. In fact, The Call Orchestration shape can be used to synchronously call an orchestration that is referenced in another project. This approach allows for reuse of common orchestration workflow patterns across BizTalk projects. When you invoke another nested orchestration synchronously with the Call Orchestration shape, the enclosing orchestration waits for the nested orchestration to finish before continuing. The nested orchestration is executed on the same thread that runs the calling orchestration.

The Start Orchestration shape is similar to the Call Orchestration shape, but in this case the nested orchestration is called in an asynchronous manner: the flow of control in the invoking orchestration proceeds beyond the invocation, without waiting for the invoked orchestration to finish its work. In order to implement this decoupling between the caller and the called orchestrations, the Start Orchestration is implemented via publication of a message to the BizTalk MessageBox. This message is then consumed by an in-process BizTalk host instance which executes the nested orchestration. When possible, use Call Orchestration, especially if the calling orchestration needs to wait for a result from the nested orchestration in order to continue processing.

Using XmlReader with XLANGMessage versus using XmlReader with XmlDocument

To improve orchestration performance for .NET methods called from an orchestration, use XmlReader with XLANGMessage, not XmlDocument. The following sample code illustrates this functionality:

// As a general rule, use XmlReader with XLANGMessage, not XmlDocument. 
// This is illustrated in the parameter passed into the following code. 
// The XLANG/s compiler doesn't allow a return value of XmlReader 
// so documents must be initially constructed as XmlDocument()
public static XmlDocument FromMsg(XLANGMessage old)
{
    //get at the data
    XmlDocument ret = new XmlDocument();

    try{
        XmlReader reader = (XmlReader)old[0].RetrieveAs(typeof(XmlReader));
        //construct new message from old
        //read property
        object msgid = old.GetPropertyValue(typeof(BTS.MessageID));
    }
    finally {
        // Call Dispose on the XLANGMessage object 
        // because the message doesn't belong to the 
        // .NET runtime - it belongs to the MessageBox database 
        old.Dispose();
    }
    return ret;
}

Another method would be to create a .NET class based on the schema. This takes less memory than loading the document into an XmlDocument object, as well as providing easy access to the schema elements for .NET developers. To generate a class based on a BizTalk schema, you can use the xsd.exe tool provided with Visual Studio 2005. For example, running xsd.exe <schema.xsd> /classes against a simple schema containing fields named ItemA, ItemB, ItemC, will produce the following class:

//------------------------------------------------------------------------------
// <auto-generated>
//     This code was generated by a tool.
//     Runtime Version:2.0.50727.1433
//
//     Changes to this file may cause incorrect behavior and will be lost if
//     the code is regenerated.
// </auto-generated>
//------------------------------------------------------------------------------

using System.Xml.Serialization;

// 
// This source code was auto-generated by xsd, Version=2.0.50727.42.
// 


/// <remarks/>
[System.CodeDom.Compiler.GeneratedCodeAttribute("xsd", "2.0.50727.42")]
[System.SerializableAttribute()]
[System.Diagnostics.DebuggerStepThroughAttribute()]
[System.ComponentModel.DesignerCategoryAttribute("code")]
[System.Xml.Serialization.XmlTypeAttribute(AnonymousType=true, Namespace="http://Schemas.MySchema")]
[System.Xml.Serialization.XmlRootAttribute(Namespace="http://Schemas.MySchema", IsNullable=false)]
public partial class MySchemaRoot {
    
    private string itemAField;
    
    private string itemBField;
    
    private string itemCField;
    
    /// <remarks/>
    [System.Xml.Serialization.XmlElementAttribute(Form=System.Xml.Schema.XmlSchemaForm.Unqualified)]
    public string ItemA {
        get {
            return this.itemAField;
        }
        set {
            this.itemAField = value;
        }
    }
    
    /// <remarks/>
    [System.Xml.Serialization.XmlElementAttribute(Form=System.Xml.Schema.XmlSchemaForm.Unqualified)]
    public string ItemB {
        get {
            return this.itemBField;
        }
        set {
            this.itemBField = value;
        }
    }
    
    /// <remarks/>
    [System.Xml.Serialization.XmlElementAttribute(Form=System.Xml.Schema.XmlSchemaForm.Unqualified)]
    public string ItemC {
        get {
            return this.itemCField;
        }
        set {
            this.itemCField = value;
        }
    }
}

This class can then be referenced in your .NET assembly in order to access the message elements, and the returned object can be directly assigned to a message. The following is an example use of the class generated above:

public static Root SetValues(Microsoft.XLANGs.BaseTypes.XLANGMessage msg)
{
   MySchemaRoot rootObj=(MySchemaRoot)msg[0].RetrieveAs(typeof(MySchemaRoot);
   rootObj.ItemA="value a";
   rootObj.ItemB="value b";
   rootObj.ItemC="value c";
   return rootObj;
}

This technique allows you to use an object-oriented approach when processing messages. This technique should be used primarily with relatively small messages because even though this technique uses considerably less memory than when loading the message into an XmlDocument object, the entire message is still loaded into memory. When processing larger messages, use the XmlReader class to read messages and the XmlWriter class to write messages. When using XmlReader and XmlWriter, the message is contained in a VirtualStream object and if the size of the message size exceeds the value specified for Large message threshold (bytes) exposed on the BizTalk Group Properties configuration page then the message is written to the file system. This decreases overall performance, but avoids out of memory exceptions.

Considerations when using maps in orchestrations

The following considerations apply when using maps in orchestrations:

  • If you are using a map to extract or set properties used with business logic in an orchestration, use distinguished fields or promoted properties. When extracting or setting values with a map the document is loaded into memory. When using distinguished fields or promoted properties, the orchestration engine accesses the message context and does not load the document into memory.

  • If you are using a map to aggregate several fields into one field, use distinguished fields or promoted properties with an orchestration variable to accumulate the result set.

  • Do not configure an orchestration with multiple inputs to transform shapes. An orchestration that contains multiple inputs to transform shapes will not stream to the file system while mapping. This behavior will cause the entire document that is being mapped to load into memory if the document size exceeds the specified TransformThreshold registry value. One possible workaround to this issue would be to apply transforms at the receive port level so that the orchestration never accepts more than a single input to transform shapes.

Impact of correlated messages on throttling

Messages that are en-queued into BizTalk Server, for example, through a receive location or by orchestrations, can be processed in one of the following ways:

  • They can activate new instances of subscribers (that is, orchestrations or send ports)

  • They can be routed to an existing subscriber instance via correlation. For more information about correlation, see “Using Correlations in Orchestrations” in the BizTalk Server 2006 R2 help at http://go.microsoft.com/fwlink/?LinkId=119927.

Many developers tend to use a single receive location to receive both activation and correlated messages for a solution through the same port. This is natural as it minimizes the number of addresses that message senders need to keep track of. However, there can be advantages to creating separate receive locations for activation messages and correlated messages.

When both activation and correlation messages arrive through the same receive location, they are subject to the same throttling state because throttling is applied at the host level. As a result, all messages arriving at receive locations in the host will be throttled as a unit. This is not an issue for many scenarios, but there are cases where throttling correlations with activations can result in a type of system deadlock.

Example

For example, consider a scenario whereby an orchestration receives an activation message that initializes a correlation set, which in turn receives a convoy of 10 messages that follow the correlation set. In addition, assume that the mix of activation and correlation messages cause a backlog of messages in the spool table which triggers a throttling condition that limits the amount of messages that can be received. Unfortunately, the correlation messages are throttled along with the activation messages, which slows down the completion of orchestrations, causing further backlog and additional throttling. Allowed to continue, this can cause the throttling mechanism to reduce system throughput to nearly zero.

By splitting the single receive location into two receive locations -- one for activations and one for correlations -- and configuring the locations in separate hosts, the database size throttling threshold for the activations can be kept lower than that for correlations, which will result in reduced overall backlog, and keep messages flowing.

So, you might be asking "Why can’t I just raise the database size threshold for my single receive location to fix the problem?" The answer is, you can, but it won’t always result in the desired behavior. Throttling is there primarily to protect the system from becoming overloaded. If you raise the thresholds high enough, or turn them off altogether, you will eliminate this protection.

Recommendation

The best practice for scenarios such as the one described above that are sensitive to throttling correlation messages, is to separate the receive locations into separate hosts which can be throttled independently.

When separate hosts are configured for receive locations, set the database size throttling threshold for the host used by the receive locations to a lower value than the database size throttling threshold for hosts used by orchestrations or correlations.

If you know that your load will never be higher than the maximum sustainable throughput (MST) for the system, or that throughput peaks are recoverable between peak events, then raising the throttling thresholds will also work, but may not sustain as high a throughput as using separate hosts for activations and correlations.

Be aware of the performance implications of using delivery notification. Delivery notification incurs overhead that negatively impacts the overall throughput of a solution so only use it when absolutely necessary. BizTalk creates an internal subscription for each delivery notification and uses an internal correlation set to return a message to the corresponding orchestration instance. More subscriptions entail more work for the master MessageBox, which ultimately reduces the maximum sustainable throughput of the system.

The two factors that have the highest impact on the performance of flat-file parsing are file size and schema complexity. An ambiguous schema is a schema that contains many optional fields. When large file sizes are used, a schema with many optional fields can degrade performance because larger files may match different branches of the schema. Schema complexity has less impact on smaller files than on larger files.

The following factors should be considered when implementing BRE in a BizTalk Server solution:

Fact types

The rule engine takes less time to access .NET facts compared to the time it takes to access XML and database facts. If you have a choice of using either .NET or XML or database facts in a policy, you should consider using .NET facts for improved performance.

Data table vs. data connection

When the size of the data set is small (< 10 or so), the TypedDataTable binding provides better performance than the DataConnection binding. However, the DataConnection binding performs better than the TypedDataTable binding when the data set is large (greater than or equal to 10 rows approximately). Therefore, you should decide whether to use the DataConnection binding or TypedDataTable binding based on the estimated size of the data set.

Fact retrievers

A fact retriever implements standard methods which are typically used to supply long-term and slowly changing facts to the rule engine before a policy is executed. The engine caches these facts and uses them over multiple execution cycles. Instead of submitting a static or fairly static fact each time that you invoke the rule engine, you should create a fact retriever that submits the fact for the first time, and then updates the fact in memory only when necessary.

Rule priority

The priority setting for a rule can range on either side of 0, with larger numbers having higher priority. Actions are executed in order from the highest priority to lowest priority. When the policy implements forward-chaining behavior by using Assert/Update calls, the chaining can be optimized by using the priority setting. For example, assume that Rule2 has a dependency on a value set by Rule1. Giving Rule1 a higher priority means that Rule2 will only execute after Rule1 fires and updates the value. Conversely, if Rule2 were given a higher priority, it could fire once, and then fire again after Rule1 fires and update the fact that Rule2 is using a condition. While this may provide a correct result, giving Rule1 a higher priority in this scenario will provide better performance.

Update calls

The Update function causes all the rules using the updated facts to be reevaluated. Update function calls can be expensive especially if a large set of rules is reevaluated when updating facts. There are situations where this behavior can be avoided. For example, consider the following rules:

Rule1:

IF PurchaseOrder.Amount > 5 
THEN StatusObj.Flag = true; Update(StatusObj)

Rule2:

IF PurchaseOrder.Amount <= 5 
THEN StatusObj.Flag = false; Update(StatusObj)

All remaining rules of the policy use StatusObj.Flag in their conditions. Therefore, when Update is called on the StatusObj object, all rules will be reevaluated. Whatever the value of the Amount field is, all rules except Rule1 or Rule2 are evaluated twice, once before the Update call and once after the Update call.

To mitigate the associated overhead, you could set the value of the flag field to false prior to invoking the policy and then use only Rule1 in the policy to set the flag. In this case, Update would be called only if the value of the Amount field is greater than 5, and the Update function is not called if the value of Amount is less than or equal to 5. Therefore, all the rules except Rule1 or Rule2 are evaluated twice only if the value of the Amount field is greater than 5.

Usage of logical OR operators

Using an increasing number of logical OR operators in conditions creates additional permutations that expand the analysis network of the rule engine. From a performance standpoint, you are better off splitting the conditions into atomic rules that do not contain logical OR operators.

Caching settings

The Rule Engine uses two caches. The first one is used by the update service and the second one is used by each BizTalk process. The first time a policy is used, the BizTalk process requests the policy information from the update service. The update service retrieves the policy information from the rule engine database, caches it and returns the information to the BizTalk process. The BizTalk process creates a policy object based on that information and stores the policy object in a cache when the associated rule engine instance completes execution of the policy. When the same policy is invoked again, the BizTalk process reuses the policy object from the cache if one is available. Similarly, if BizTalk process requests for the information about a policy from update service, the update service looks for the policy information in its cache if it is available. The update service also checks if there have been any updates to the policy in the database every 60 seconds. If there are any updates, the update service retrieves the information and caches the updated information.

There are three tuning parameters for the rule engine related to these caches, CacheEntries, CacheTimeout, and PollingInterval. You can specify the values for these parameters either in the registry or in a configuration file. The value of the CacheEntries parameter is the maximum number of entries in the cache and is set to a value of 32 by default. You may want to increase the value of the CacheEntries parameter to improve performance in certain scenarios. For example, say you are using 40 policies repeatedly; you could to increase the value of the CacheEntries parameter to 40 to improve performance. This would allow the update service to maintain cache details of up to 40 policies in memory.

The value of CacheTimeout is the time in seconds that an entry is maintained in the update service cache. In other words, the CacheTimeout value refers to how long a cache entry for a policy is maintained in the cache without being referenced. The default value of CacheTimeout parameter is 3600 seconds, or 1 hour. It means that if the cache entry is not referenced within an hour, the entry is deleted. In some cases, it may be beneficial to increase the value of the CacheTimeout parameter to improve performance. For example if a policy is invoked every 2 hours performance of the policy execution would be improved by increasing the CacheTimeout parameter to a value higher than 2 hours.

The PollingInterval parameter of the rule engine defines the time in seconds for the update service to check the rule engine database for updates. The default value for the PollingInterval parameter is 60 seconds. If you know that the policies do not get updated at all or are updated rarely, you could change this parameter to a higher value to improve performance.

SideEffects property

The ClassMemberBinding, DatabaseColumnBinding, and XmlDocumentFieldBinding classes have a property named SideEffects. This property determines whether the value of the bound field, member, or column is cached. The default value of the SideEffects property in the DatabaseColumnBinding and XmlDocumentFieldBinding classes is false. The default value of the SideEffects property in the ClassMemberBinding class is true. Therefore, when a field of an XML document or a column of a database table is accessed for the second time or later within the policy, its value is retrieved from the cache. However, when a member of a .NET object is accessed for the second time or later, the value is retrieved from the .NET object, and not from the cache. Setting the SideEffects property of a .NET ClassMemberBinding to false will improve performance because the value of the field is retrieved from the cache from the second time onwards. You can only do this programmatically. The Business Rule Composer tool does not expose the SideEffects property.

Instances and selectivity

The XmlDocumentBinding, ClassBinding, and DatabaseBinding classes have two properties: Instances and Selectivity. The value of Instances is the expected number of instances of the class in working memory. The value of Selectivity is the percentage of the class instances that will successfully pass the rule conditions. The rule engine uses these values to optimize the condition evaluation so that the fewest possible instances are used in condition evaluations first and then the remaining instances are used. If you have prior knowledge of the number of instances of the object, setting the Instances property to that value would improve performance. Similarly, if you have prior knowledge of the percentage of these objects passing the conditions, setting the Selectivity property to that value would improve performance. You can only set values for these parameters programmatically. The Business Rule Composer tool does not expose them.

This section describes performance metrics that should be evaluated when measuring the performance of a BizTalk solution. This section provides pointers to existing documentation as a reference when more information is needed for a specific performance object or counter that might be mentioned later in this document.

BizTalk Server performance metrics

For a detailed description of the performance counters available in BizTalk Server and to get information about counters mentioned in this document, see “Performance Counters” in the BizTalk Server 2006 R2 help at http://go.microsoft.com/fwlink/?LinkID=104641.

SQL Server performance metrics

Because SQL Server performance is such an integral part of the overall performance of a BizTalk Server solution it is important to have a thorough understanding of how to evaluate and troubleshoot SQL Server performance. For more information on evaluating and maximizing the performance of the SQL Server computers that host the BizTalk Server databases, see the following topics:

Was this page helpful?
(1500 characters remaining)
Thank you for your feedback
Show:
© 2014 Microsoft