The best way to persist incoming data from a BizTalk process into BAM is to use the Tracking Profile Editor (TPE). For data coming from a non-BizTalk process, use the BAM APIs to send the data to the BAM Primary Import database.

The Tracking Profile Editor is a graphical user interface to help developers create new tracking profiles or modify existing tracking profiles for the BizTalk engine. Specifically, the TPE tool is used to drag items from orchestrations, messages, or pipelines and map them to the data items in an existing activity. An activity is deployed in the BAM Primary Import database. The TPE can drag items from orchestrations or messages and attach them to an activity. The data about individual activity instances is available for queries in a dynamically created SQL view in the BAM Primary Import database.

To extract BAM information, use a BAM view rather than going directly to the SQL tables in the Primary Import database. The view aggregates information from several tables to surface a complete view of the data. Additionally, the actual table structure may change and affect the created view in future releases.

For more information about querying BAM instance data see:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/BTS06CoreDocs/html/ae4a8854-d5c2-4b36-a0ef-3f74e138306e.asp

You can find information about the BAM TPE and BAM API at these respective locations:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/BTS06CoreDocs/html/064f5795-28d4-40c5-a0d2-295d3516d7f3.asp

and

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bts06CoreDocs/html/32a925f2-c7f4-4111-9c59-8865f15c6a89.asp

Clicking the TPE Select Event Source button (see the following figure) brings up the following options:

• Select Orchestration Schedule
  
• Select Messaging Payload
  
• Select Context Property
  
• Select Messaging Property
  

If you want to track anything from an orchestration, you must select the first option, Select Orchestration Schedule. The remaining three options are used only if you want to track from messaging (pipelines and ports).

For orchestration tracking, after you have selected the appropriate orchestration schedule, you can perform the mapping options as follows:

• Milestone mapping. Select an orchestration shape in the right pane and drag the shape to the corresponding activity milestone in the left pane.
  
• Non-milestone mapping. Select a shape for the context and right-click it. Click one of the following:
  
  • Message Payload Schemas
    
  • Context Property Schemas
    
  • Message Property Schema
    

Pick the appropriate schema item in the right pane and drag it to the corresponding activity item in the left pane.

The following figure shows a mapping from an orchestration schedule. By right-clicking the Receive1 shape context you can select your choice for mapping to non-milestone data.

When using the TPE main window to create a tracking profile for a BAM activity that spans multiple ports and orchestrations, the following two issues may arise.

Difficulty in Specifying Ports for Data Items Tracked from Ports

Continuation and ContinuationIDs

This executable file controls BAM tracing. You can use it in two ways:

• At a command prompt
  Trace:on|off—Switch tracing on or off, overriding the configuration settings. This is coupled with any BM.exe command. For example:
  bm.exe deploy-all -DefinitionFile:PO.xml –Trace:On
  

• In the BM.exe.config configuration file
  This file contains a section where you can enable tracing. By default, tracing is not enabled. To enable tracing, remove the comments around the following lines of code:
  <switches>
  <add name="bm" value="1" />
  <add name="Microsoft.BizTalk.Bam.Management" value="1" />
  </switches>
  

The BAM Event Bus service is also called the Tracking Data Decode service (TDDS). To turn on tracing for the BAM Event Bus, edit the following section of the BTSNTSvc.exe.config file:

<system.diagnostics>

<switches>

<add name="Microsoft.BizTalk.Bam.EventBus" value="1" />

</switches>

<trace autoflush="true" indentsize="4">

<listeners>

<add name="Text" type="System.Diagnostics.TextWriterTraceListener" initializeData="c:\tdds.log"/>

</listeners>

</trace>

</system.diagnostics>

The BAM Portal is an ASP.NET application, and follows the standard protocol for tracing. Within the web.config file, there is a section called <trace> that you can enable. For example:

<trace enabled="true" localOnly="true" pageOutput="false" />

The BAM Alerting system uses SQL Server Notification Services for the alerting infrastructure. To turn understand better how to configure SQL Server Notification Services, see the following link:

http://msdn2.microsoft.com/en-us/library/ms172486.aspx

When the Tracking Data Decode service (TDDS) fails to process tracking or BAM data, it writes the failure information to the TDDS_FailedTrackingData table in binary format. This table is part of the schema of the BAM database. (This database is named BAMPrimaryImport by default, and may also be referred to as the PrimaryImport database, the PIT, or the PIDb). The HAT database is the Tracking database (referred to also as the DTADB database).

You may receive an error message that is similar to the following in the application log of a server that is running Microsoft BizTalk Server:

Reading error. Exception information: TDDS failed to read from source database. SQLServer: SQLSERVER, Database: BizTalkMsgBoxDb.

Or in the event log you may see:

Either another TDDS is processing the same data or there is an orphaned session in SQL server holding TDDS lock. Either another TDDS is processing the same data or there is an orphaned session in SQL server holding TDDS lock. SQL Server: ServerName Database: DatabaseName

For more information about these messages, see KB article http://support.microsoft.com/?scid=kb;en-us;841334&spid=1444&sid=312

If a BizTalk Server installation is initially done without BAM, and later BAM is installed and configured, the BizTalk Server Application service (the BizTalk Tracking Host) needs to be restarted so that its TDDS subservice can pick up the new connection string. Otherwise the Event Bus service reports an error in the Event log that a stored procedure is not found.

If you check the connection string (in the BizTalkMgmtDb database, TDDS_Destinations table, ConnectionString column where DestinationName = "Business Monitoring") you will find that it is pointing to the BAMPrimaryImport database. However, the error message indicates that the events that failed to be processed were moved to the TDDS_FailedTrackingData table in the BizTalkDTA database. This error message indicates that the connection strings encountered a timing issue during configuration, which means that the update of the original connections is still in the cache and requires recycling of the Event Bus service (the BizTalk Host under which the service is running).

To fix this problem, restart the BizTalk Server Application service. Because BAM was installed after the initial BizTalk Server installation, the connection string must be changed by the restart process. 

For information about using the BAM Management utility, see:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/BTS06CoreDocs/html/c55aabe2-f38b-4917-863c-b408a4eef98e.asp

For information about backing up and restoring BAM databases, see:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/BTS06CoreDocs/html/5d477492-fdb7-4866-92a8-2720fea15839.asp

After data is transferred out of the BAMPrimaryImport database into the BAMArchive database that data is no longer available to BAM. Retrieving the data would require custom work for search or historical comparison. Because of this, the BAMArchive database is not part of the BizTalk Server backup process.

The BAM Data Maintenance DTS Task works by activity. You probably want to schedule all DTS task to run serially one time per period and to write only complete newly archived tables. There is never any need to back up multiple times between data maintenance DTS tasks. You can copy whole tables to long-term storage, process data into a data warehouse, or use another method for backup

This BAMArchive database is used to store old business activity data. The goal of creating a BAM Archive database is to minimize the buildup of business activity data in the BAM Primary Import database. It is best to back it up after the DTS task completes

You can create BAM views based on data stored in an online analytical processing (OLAP) cube. The Cube Update Data Transformation Services (DTS) package refreshes the data in the cube so that OLAP views reflect the correct data.

The basic idea of activity data storage is to have a separate table for each activity type, in which each record represents a different activity instance (for example, in-progress or completed).

DTS packages were renamed to SSIS packages in SQL Server 2005. These tasks are used to accomplish activities that should be executed regularly. For each activity there is one task for data management and another task for moving data to the analysis server per view.

For information about scheduling DTS packages, see:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/operations/htm/ebiz_ops_bam_admin_eqom.asp

You can find additional information at:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/BTS06CoreDocs/html/bba70d81-6ddf-4f1f-a1f7-d5a5bf453bae.asp

There is no public documentation for the BAM Query Web service (QWS). It is not a supported interface and is merely our internal implementation of our own BAM portal. We also do not support this because we expect the format of queries to change with versions of the product and do not want to create a migration issue for our customers. We recommend that customers avoid using the QWS.

Rather, the supported and documented way of extracting instance data from BAM is to connect directly to Microsoft SQL Server™ and query the views installed by BAM. The data about individual activity instances is available for queries in a dynamically created SQL view in the BAM Primary Import database. The process is documented at this location:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/BTS06CoreDocs/html/780aa6d4-0cb4-4aee-949e-803b10dec4de.asp