BizTalk Orchestration Example: Automating the Procurement Process
Summary: This white paper will show an example of how to use BizTalk Orchestration and COM+ to solve a common real-world business requirement. (13 printed pages)
The purpose of this white paper is to illustrate usage of various elements of Microsoft® BizTalk™ Server 2000. It highlights the following key technologies:
- Automatic schedule instantiation
- Importing flat files
- BizTalk Orchestration timed transactions
- Schedule correlation with a non-instantiating COM component
- BizTalk Orchestration while loops
- Dynamic document routing
This white paper is based on a procurement process business scenario. Requests for quotation (RFQs) for a particular commodity item are sent out to three suppliers. The suppliers send back their quotes and users issue an order to the lowest bidder.
The procurement process is divided into two logical business processes:
- The RFQ process of requesting, receiving, and processing quotations (an XLANG schedule sends out the requests for quotation and selects a winning bidder).
- The purchase process of actually placing orders (an XLANG schedule sends out an order using the BizTalk Server self-routing document capability).
The following illustration shows the entire procurement cycle, which consists of two XLANG schedules. The illustration shows the tasks that the schedules contain and the evolution of the data as it moves through those tasks.
The top band shows the two BizTalk Orchestration XLANG schedules. These are stand-alone schedules that are loosely coupled through Message Queuing (also known as MSMQ).
The second band shows the high-level tasks that occur in the XLANG schedules.
The third band shows the names of the documents as identified using BizTalk Messaging Manager.
The fourth band shows the structure of the documents at the various stages in their life cycle, and shows where in the process flow the documents evolve from one type of document to another.
The RFQ process sends out requests for quotation to three suppliers, waits for a predetermined period of time, and selects the lowest bidder as the winner. At a high level, we will follow these steps:
- Convert a flat file requisition to an XML message
- Instantiate the schedule
- Send the RFQ to three suppliers
- Wait for either three responses or a time-out, whichever occurs first
- Place the winning bid on a Message Queuing queue
BizTalk Messaging Entities
The entities we will need in BizTalk Messaging to support our ProcessRFQs XLANG schedule are as follows:
|ReceiveRequisition||Messaging port||Activation port to our ProcessRFQs XLANG schedule|
|Messaging ports||Ports to the three suppliers used in this scenario|
|ReceiveResponse||Messaging port||Non-instantiating COM port used to receive quotation responses from the suppliers|
|SendRFQs||Distribution list||Distribution list that points at the three supplier ports|
|ReceiveRequisition||Channel||Channel called by the receive function as part of schedule instantiation|
|SendRFQs||Channel||Channel to the distribution list of bidders|
|Document definitions||Documents that we will flow through the business process|
|Maps||Maps that are used to convert between document types|
The entire procurement process is triggered by a tab-delimited flat file appearing in a folder that is being polled by a BizTalk Server receive function. The file is deposited into the folder by an external process, perhaps an export operation from another application, an FTP transfer, creation from an ASP page, or even a file copy operation. The following is a sample file:
Figure 2. Sample file
BizTalk Server needs to know the structure of a flat file in order to parse it properly. This is achieved by using an envelope, which sets out the structure of the ASCII file and serves as a wrapper for the document as it moves through BizTalk Server. Delimiters for the flat file are specified in BizTalk Editor on the Parse tab.
The Advanced tab of the file receive function allows you to specify which envelope should be used (if any), as well as a channel. In our case, we have an envelope called Requisition and a channel called ReceiveRequisition.
The final step in importing the document is to let BizTalk Server know how to convert the ASCII file to the document definition set forth by the envelope, which is done by the FlatfileToRequisition map.
At this point, BizTalk Messaging Services now has enough information to be able to pick up the flat file and send it to the channel. The map is applied as the message flows through the channel. The document that emerges is an XML document, a Requisition.
Into the Schedule
After the requisition has been passed on by the receive function, the remainder of our business process is managed by BizTalk Orchestration. The following is the XLANG schedule for this process, as seen from BizTalk Orchestration Designer:
At a high level, the following steps occur in the schedule:
- Receive a requisition.
- Add a reference to this instance of the schedule.
- Send the RFQ out to three suppliers.
- Enter a timed transaction and wait for three responses or a time-out (whichever occurs first).
- Determine the lowest bidder.
- Place the RFQ returned from the lowest bidder on a Message Queuing queue.
We will now look at each of these steps in detail.
Receiving a requisition
We have seen that a receive function polls a folder for a flat file and pushes it through a channel that maps it to a Requisition XML document.
The ReceiveRequisition channel called by the receive function is connected to a ReceiveRequisition messaging port that has a new XLANG schedule specified as a destination with a port name of ReceiveRequisition. When the channel is invoked, a new instance of the schedule will be created and the received document will be passed into it at the ReceiveRequisition port.
Adding an instance reference
As the business process runs and we send out RFQs, we will in all likelihood have multiple instances of this schedule in progress at various stages of their life cycle and at any given time. We need a mechanism that will allow us to correlate each response that we receive back to the schedule instance that originated the corresponding RFQ.
Each instance of an XLANG schedule has a unique identifier associated with it (a GUID). This GUID can be derived from a port reference.
In this example, we pass our RFQ document to a Windows® Script Component that builds a URL to an Active Server Pages (ASP) page that will act as the response recipient. The URL consists of two distinct parts: the ASP page that will receive the response, and a parameter, wfid, which specifies the GUID of the schedule instance. We build this Reply-to URL dynamically in the script component and it will look something like this:
With the moniker value assigned to wfid, we will have everything we need to make the correlation between the originating requests and the responses we receive from our suppliers.
Sending out RFQs
For the sake of this sample, we have set up three fictitious suppliers, each of which has a BizTalk Server messaging port set up to receive the RFQs. The ports point at RFQ.asp on localhost. The sole purpose of RFQ.asp is to receive an RFQ document, populate the price with a random value, and then post the document to the location specified in the Reply-to element.
We could have sent out the RFQs individually to each supplier, but for the sake of convenience, we have set up in BizTalk Messaging Manager a BizTalk Server distribution list that includes each of these three messaging ports. This allows us to send out all three RFQs at one time.
Receiving responses—the RFQs component
The RFQs COM component contains two classes:
- A loop iteration counter
- An RFQ assessor
The loop iteration counter is used to count the number of passes through the message receiving loop, and the RFQ assessor captures the returned quotations and exposes a method to determine the lowest bidder.
There may be many concurrent instances of this schedule in progress at any point in time. This leaves us with a dilemma. We need to maintain state information (the iteration count through the loop and any messages received as part of the loop), yet doing so would require the server to keep the stateful resources alive as it waits for responses. This is not a viable solution because it would not scale.
The solution is to let BizTalk Server dehydration persist the stateful components.
For the XLANG Scheduler Engine to be able to persist a COM component, it must support the IPersist interface. To do this with a Microsoft Visual Basic® component such as the RFQs component, select "Persistable" in the properties of the class in the Visual Basic IDE.
The XLANG Scheduler will not be able to dehydrate schedules that host stateful components that do not support IPersist, so this situation should be avoided if there are considerations around scalability or server resources.
Receiving responses—the timed transaction
From a business perspective, there is a special consideration in our scenario that must be taken into account: although we sent out three requests for quotation, we don't know how many responses we will receive. We can request a receipt for the message and use a receipt channel to confirm that the message was received, however that only tells us that the RFQ was received and does not necessarily mean that the supplier will be sending a reply.
The solution to the non-responding bidder issue is to use a timed transaction, and set the transaction time-out to be the time to wait before moving on.
BizTalk Orchestration Designer uses colored regions to indicate transaction boundaries. These regions are color-coded according to transaction type, and in the case of a timed transaction are blue as shown. Note that in the schedule we refer to our COM component both before and after the transaction. We need to select the winning bidder after the transaction has completed or aborted, because we have two potential exit conditions: receiving all the responses or a transaction time-out. If we create the RFQ assessor inside the transaction, it will not be available to us if the transaction fails. For this reason, we have an Initialize stub method that is called to force the XLANG Scheduler to create the component before entering the transaction.
Figure 4. Receiving quotes
Receiving responses—the while loop
Responses are received inside of a while loop in the schedule.
As you would expect, BizTalk Server while loops continue processing a specified set of tasks while a specified condition is true. A while shape has two exit branches: the "Continue" branch that is followed when the loop check condition is not true, and the loop branch that leads to the tasks that make up the loop itself. This loop branch is terminated with an End shape—which when executed indicates the end of the branch, not the end of schedule execution—and control returns to the loop check condition.
In our case, the check condition is the counter in the RFQs component. However, we have a sequencing issue to resolve: the loop continuation condition (a Boolean value) needs to be initialized before we test it, or it will be null as far as the XLANG Scheduler is concerned and we will never enter the while loop. This is the reason for making sure the Start Counter task immediately precedes the while loop.
After we enter into our loop, we increment the iteration counter and then wait for a response to be received. If three responses have been received prior to the time-out, the loop condition is no longer true and we exit the loop.
Receiving responses—the non-instantiating COM object
XLANG schedules can contain a special kind of COM component, a non-instantiated COM component that serves as a point you can refer to from outside the schedule. The wfid parameter we added to our Reply-to URL is actually a moniker, and by appending the name of the non-instantiating COM component port, we will be able to get a reference into the schedule. Conceptually, this is a one-way path from an outside application into a running XLANG schedule instance, and this is how we will ultimately be able to correlate a response with the schedule instance that originated it.
In the sample application, we use an ASP page, ReceiveQuote.asp, to receive the incoming quotations. To clarify this topic further, relevant code from the page is shown below:
'// Read the workflow ID parameter. Must do this before doing the binary read. strWorkflowID = Request("wfid") '// Create a moniker using the workflow instance ID and the port reference strMoniker = strWorkflowID & "/ReceiveDoc" '// Create an instance of the XMLDOM to hold the incoming response Set xmldoc = Server.CreateObject("MSXML2.DOMDocument") xmldoc.async=false '// Do a binary read of the XML that was POSTed to us xmldoc.load(Request) '// Create an instance of the support object Set objPort = Server.CreateObject("XLANGSupport.Correlate") '// Submit the document we received Call objPort.SubmitToPort(strMoniker, xmldoc.xml)
ReceiveQuote.asp receives the response that was posted by HTTP from one of our bidders, loads it into an instance of the XMLDOM and instantiates an XLANGSupport.Correlate component that is used as part of the correlation process (included with the source code for this white paper). We then call the submitToPort method of the XLANGSupport.Correlate component, passing through the moniker and received document, as shown below:
Public Function submitToPort(strMoniker, strDoc) Dim objPort Set objPort = GetObject(strMoniker) objPort.ReceiveDoc (strDoc) Set objPort = Nothing End Function Public Function ReceiveDoc(strDoc As String) '// shell required by BTS port binding wizard. Do not remove... End Function
There are three ways to use the GetObject function to work with schedules:
- When you issue a GetObject call passing through a moniker that includes the name of a schedule, a new instance is created and a reference to it is returned.
- When you issue a GetObject call passing through a moniker that includes an instance GUID for a schedule, a reference to that running schedule instance is returned.
- When you issue a GetObject call passing through a moniker that includes an instance GUID for a schedule, and also a port name, a reference to that port in that running schedule instance is returned.
The port name refers to a non-instantiating COM object in the schedule. We are effectively calling a method that only accepts a single parameter, which is a message being injected into a running schedule instance. A stub method must exist in a COM component in order for the COM Component Binding Wizard in BizTalk Orchestration to identify the port, although no code will be executed. The port is just an entry point to our schedule.
In this case, the GetObject(strMoniker) call will get us a reference to the ReceiveDoc port of the running instance of the schedule identified by the GUID in the moniker. We then call the ReceiveDoc stub function on the port to pass in the received document.
So, to recap the route back into the workflow from our bidder:
- Message is received by ReceiveQuote.asp.
- ReceiveQuote.asp instantiates XLANGSupport.Correlate.
- ReceiveQuote.asp calls the SubmitToPort method of XLANGSupport.Correlate, passing through a moniker containing instance and port information, as well as the document being submitted.
- SubmitToPort gets a reference to the specified workflow instance.
- SubmitToPort calls the "ReceiveDoc" method of the port.
The XLANG Scheduler resumes execution of the workflow at the ReceiveQuote port. The only input value is the document that was received.
The last step in our quotation receive process is to save a copy of the document in our COM component, which is the RetainQuote task that passes the document it just received to the AddQuote method of the RFQs component.
Determining the lowest bidder
As we iterated through the while loop that received quotations from our suppliers, we were collecting those quotations inside a stateful COM component.
It is very important for performance reasons to specify the statefulness of components in the Orchestration COM Component Binding Wizard. This is where you are telling the XLANG Scheduler whether your component maintains state information and if so, whether it can be persisted.
The GetBestQuote task calls the GetBestQuote method of the RFQs component, which will return the lowest priced quotation.
Placing the winning bid on a Message Queuing queue
The ProcessRFQ schedule communicates with the ProcessOrder schedule through a message queue. The end result of our Request for Quotation business process, the winning bid, gets placed on an MSMQ message queue.
This loosely coupled message-based architecture affords us the most scalability and flexibility.
Connecting Up the Data Flow
The following diagram shows the message flow through the schedule.
Item 1 shows the call to the Windows Script Component. Note the Port Reference to AddWFID. It is from this value that we are able to extract the instance GUID that we use for correlation. The output of the AddWFID script is the RFQ message, Item 2.
Item 3 shows the quotation we received being passed through to the AddQuote method call. Item 4 shows the result of the GetBestQuote flowing to the OrderToBePlaced message that we will put on the message queue.
Figure 5. Quotation in the AddQuote method call
The second workflow involved in this example is far simpler than the first, because the business process itself is far simpler. All we do here is pick up a winning bid from a known queue location, convert it to a purchase order, and send it out to a vendor.
This schedule features an important capability of BizTalk Messaging: dynamic routing. The endpoint to which we send the purchase order is specified in the document itself.
BizTalk Messaging Entities
The entities we will need in BizTalk Messaging Services to support our PlaceOrder XLANG schedule are as follows:
|WinningBid||Messaging port||Activation port to the PlaceOrder XLANG schedule|
|Order||Messaging port||Open destination messaging port used to send orders|
|WinningBid||Channel||Channel called by the receive function as part of schedule instantiation|
|Order||Channel||Channel used to send outbound order|
|Purchase Order||Document definition||Purchase order sent to a supplier|
|RFQToPurchaseOrder||Map||Converts an RFQ to a purchase order|
The purchase order issuing process is triggered when an RFQ appears in a Message Queuing queue that is being polled by a BizTalk Server receive function. In our case the message is being deposited by the ProcessRFQs XLANG schedule that we just discussed, but other processes or applications could also be depositing messages there.
Into the Schedule
After the RFQ has been passed on by the receive function, the remainder of the business process is managed by BizTalk Orchestration. The following is the XLANG schedule for this process, as seen from the BizTalk Orchestration Designer:
You can see that this schedule is far simpler than the first one. These are the steps that occur in the schedule:
- Receive RFQ
- Display a message
- Send an order
We will now look at each of these steps in detail.
Receiving an RFQ
In the prior example, we saw that a receive function can poll a file folder. In this case we're polling a Message Queuing queue instead.
When an RFQ appears in this predefined queue, the receive function pushes it through the WinningBid channel. As it flows through the channel, the RFQToPurchaseOrder map is applied and the RFQ document is converted into a PurchaseOrder document.
The WinningBid channel called by the receive function is connected to a WinningBid messaging port that has a XLANG schedule instantiation specified as a destination, with a port name of GetWinningBid. This means that when the channel is invoked, a new instance of the PlaceOrder schedule will be created and the received document will be passed into it at the GetWinningBid port.
Display a message
Obviously we would not have this situation in the real world. This task simply provides feedback as the sample runs because this is the only feedback mechanism in either of the schedules in this white paper. It displays the endpoint of the winning bidder.
Send an order
We have now reached the end of this business process that began with a simple flat file. This is also where we see the dynamic routing at work.
When we reach this task we call the submitOrder method of our RFQs component, passing the document into it.
The submitOrder method will take that document and submit it to BizTalk Messaging, which in turn will send it to the proper destination. The destination endpoint is included in the RFQ document with which we started this business process—in the OrderAccepterAddress element. But how does BizTalk Server know which element to use?
The answer is that we have told it where to look, by using BizTalk Editor to specify in the document definition that the OrderAccepterAddress element is the "Destination Value." This identification is done on the Dictionary tab, as shown below:
Figure 7. Identifying the destination value
This white paper has shown an example of using BizTalk Orchestration and COM+ to solve a common real-world business requirement.
From a business standpoint, we have created a process whereby a flat file enters one end of a business process, it gets sent to multiple suppliers as a request for quotation, and the successful bidder receives an order. The entire process is highly automated and runs with no human intervention whatsoever.
From a technical standpoint, we have exercised many capabilities of BizTalk Server. We have seen:
- How to use while loops and timed transactions in BizTalk Orchestration.
- How to use a non-instantiating COM component to correlate a response with a running schedule instance.
- How separate business processes can be loosely coupled together using Message Queuing.
- How the dynamic self-routing document capabilities of BizTalk Server can be used to route a document to a destination specified inside the document itself.
This white paper has exercised some of the more advanced capabilities of BizTalk Messaging and BizTalk Orchestration, and has shown how they can be assembled to provide advanced business process automation solutions.
Brian Loesgen is a Principal Software Engineer at Stellcom Inc., a leader in providing wireless-based engineering solutions for the mobile economy. Brian utilizes his XML expertise to translate new, leading-edge technology into real-world value. He is a co-author of the "Professional XML," "Professional ASP/XML," and "Professional Windows DNA" books from Wrox, as well as having written technical white papers for Intel, Microsoft and others. Brian has spoken at XML One, XML Devcon Conferences, the Wrox Developer Conferences, and numerous other major technical conferences worldwide.