Calendar Connector Architecture

Calendar Connector supports synchronization of free/busy information between Exchange Server 2003 and Lotus Notes or Novell GroupWise, so that users in these messaging systems can query each other's free/busy information when they create meeting requests. The requirements for Calendar Connector are similar to those for Connector for Lotus Notes and Connector for Novell GroupWise. These connectors must be installed in the same administrative group as Calendar Connector and should be configured before Calendar Connector. For information about how to install and configure Calendar Connector, see the Exchange Server 2003 Interoperability and Migration Guide.

Free/Busy Information

Free/busy refers to certain information published by a messaging client for a user. This information consists of the user's personal availability data determined by the contents of the Calendar folder in their mailbox. As expected, free/busy data is used extensively when scheduling meetings. Free/busy data is stored as messages in a dedicated system public folder. Each administrative group in the Exchange organization includes a Free/Busy folder.

You must install Calendar Connector on a server that is running Exchange Server that contains a replica of the free/busy system folder for the administrative group. Free/busy data can be replicated within the Exchange organization to any one of the public folder servers, or it can reside on a single server that runs Exchange Server. Within the organization, the free/busy data is replicated using standard public folder replication.

You can check whether a server that runs Exchange Server contains a replica of the free/busy system folder for the administrative group. For detailed instructions, see How to Check Whether a Server Running Exchange Server Contains a Replica of the Free/Busy System Folder for the Administrative Group.

Publishing of Free/Busy Data

The publishing of free/busy data is a client operation performed by Outlook and Outlook Web Access. The Exchange store is not involved in this process, with the exception of a system public folder in a public folder store that is used for storing and publishing data.

Outlook first gets a referral from the mailbox server for the associated Free/Busy public folder. After the server is located, properties on the user object in Active Directory are used to find the free/busy message in the public folder.

Free/Busy Messages

Each free/busy message is a representation of the days and times that are busy and the days and times that are not busy for a single person or resource. This is represented by a stream of numbers on the Free/Busy server (a public folder server with public folders containing replicas of one or more of the Free/Busy site folders).

One representation is 002222000033333333, where each number represents X minutes of increment (as specified in the request, with 6 minutes being the lowest granularity). This interpretation of the numbers is discussed in the following table.

Free/busy messages

When there are multiple appointments in a single timeslot, the appointment with the highest status number is selected. For example, a slot that contains both an OOF (3) appointment and a tentative (1) appointment is coded as OOF (3).

More specifically, free/busy data is stored in multiple groups of multi-valued integer arrays. Each group represents one of the busy classifications (busy, tentative, or OOF), and each item in the group represents one month of data. The array itself is a group of pairs, in which each pair is the number of minutes into the month the busy period starts and ends, time-zone-adjusted to the International Date Line. Therefore, no data is stored for free time, because free time is implicitly computed as being all of the time that is not marked as busy.

The appointment also stores the start month and month count, so that clients can compute appropriately.

Free/Busy Data Generation

There are various ways to generate free/busy data. For example, Outlook modifies a user's free/busy items as calendar items are saved, and periodically publishes this message to their server that is running Exchange Server, using MAPI. Outlook Web Access and Outlook Mobile Access clients also generate free/busy items as calendar items are saved. From there, Outlook Web Access or Outlook Mobile Access sends the message through collaboration data object (CDO) and WebDAV to System Attendant, which is responsible for additional processing the message and publishing to the server that is running Exchange Server.

Free/Busy Publishing in Outlook

Outlook publishes free/busy data for a user periodically (by default every 15 minutes), and upon shutdown. Every time that free/busy information is published, the entire free/busy item is updated (not only the changes). The user may specify the number of months of future free/busy information to publish on the server. Two months is the default, and 36 months is the maximum duration. By default, the free/busy data is published for one month in the past.

When Outlook must publish, it first determines the folder to which to publish free/busy data, based on the legacyExchangeDN of the user. The legacyExchangeDN consists of two parts. The first part determines the path of the free/busy folder (also the administrative group in which the user was created, because the legacyExchangeDN does not change when moving user mailboxes between administrative groups), and the second part is the subject of the free/busy message. For example, the free/busy data location for a user who has the legacyExchangeDN of /o=Contoso/ou=CorpUsers/cn=Recipients/cn=UserName is the folder "EX:/o=Constso/ou=CorpUsers," and the message has a subject of "User-/cn=Recipients/cn=UserName."

The free/busy folder is a subdirectory of the Schedule+ Free Busy folder, under the NON_IPM_SUBTREE. The message subject is USER-/cn=RECIPIENTS/cn=UserName. If a duplicate free/busy message is created, the information store automatically appends the suffix -2 to the URL of the message. Therefore USER-/cn=RECIPIENTS/cn=UserName becomes USER-/cn=RECIPIENTS/cn=UserName-2. This duplication of messages is not common, but it can occur because of client errors, replication failures, and so on. If Outlook discovers duplicate entries for a user while publishing data, it deletes them. The System Attendant's free/busy publishing agent also deletes duplicate entries when it is updating free/busy information about behalf of Outlook Web Access or Outlook Mobile Access.

After Outlook determines where to publish the message in the public folder store, it chooses a free/busy server. The steps are as follows:

1. Upon first logon, the server indicates to the client which default hierarchy server to contact. This information is stored in the user's profile. If the administrator changes the setting in Exchange System Manager, the user must log out and log back in to get the new value. The client then makes an initial connection to this server and maintains the connection for the duration of the user's logon session.

2. The MAPI client requests a "hierarchy" table for the root of the public folder store. This request is sent to the configured default public folder store, and folders stored at the root level of the public folder store are returned to the client.

3. The client enumerates the folder entries in this table, looking for the folder of interest. If it is required, the client subsequently opens subfolders, opens their table of sub-subfolders, and enumerates the subfolders again.

4. After the MAPI client identifies the folder of interest, it requests the table of contents for that folder.

5. The service provider queries the server for the list of content replicas for the folder.

6. The server obtains the replica list for the folder from the database and sorts it based on routing group connector costs from that server. Other content replicas in the same routing group as the requested server have a connector cost of zero.

7. The sorted list is returned to the client, together with the number of items in the group of lowest-cost servers.

8. If the server to which the client is already connected is in the replica set (its cost is also zero), the content replica search is finished. Go to Step 10.

9. The user's legacyDN is hashed, and the hash result is then divided by the count of the lowest-cost servers. The rest of the division is used to index the list of servers returned and to pick a server to which to connect.

10. The service provider tries a connection to the chosen server. If the connection succeeds, the entire process is finished, and the server returns the contents table to the MAPI client.

11. If the connection fails or the server reports "I'm not a replica" (the replica set might have changed, and that change might not yet have replicated to the server from which the replica list came), the service provider removes this server from the list, decrements the count of "lowest-cost" servers, and if that count is not at zero, returns to Step 9.

12. If the list of lowest-cost servers is exhausted, the count is reset to the size of the remaining servers in the list, and the process returns to Step 9. If the entire list is exhausted, an error is returned to the MAPI client.

Free/Busy Publishing in Outlook Web Access and Outlook Mobile Access

Because they do not use MAPI, Outlook Web Access and Outlook Mobile Access cannot publish free/busy data directly to the public folder store. Instead, they rely on a free/busy publishing agent (MadFB) that runs in the System Attendant process (Mad.exe).

MadFB has two functions:

• Publishing free/busy messages for Outlook Web Access and Outlook Mobile Access

• Deleting duplicate free/busy messages

As part of the transaction involved in the creation, deletion, or update of an appointment that affects the start or end time, a server-side call sends a free/busy update message to the System Attendant's mailbox. MadFB, which resides inside System Attendant, processes these messages and updates the free/busy data in the MAPI public folder. Depending on System Attendant's message polling interval, there can be up to a 15-minute delay before the updated free/busy data is published.

MadFB's publishing process is identical to the Outlook publishing process described earlier. Therefore, if duplicate messages are present, they are appended by a number. Although Outlook Web Access and Outlook Mobile Access follow a process that is similar to the process that Outlook follows, the Outlook Web Access and Outlook Mobile Access processes are generally more reliable, because all the processing occurs between servers that are running Exchange Server.

Free/Busy Data Lookup

When locating free/busy data, Outlook operates differently than Outlook Web Access and Outlook Mobile Access, as described in the following bullets However, for all clients, this process involves first locating the free/busy public folder, and then accessing a particular user's free/busy data in the folder.

• Outlook   Before Outlook locates the free/busy public folder, it first receives a referral from the mailbox server for the associated public folder store, which the free/busy server then queries against (the referral and querying process is similar to the publishing process). The free/busy data is stored as messages in the site folder that is located in the main free/busy folder. Outlook, using Active Directory and Exchange Server, determines a user's legacyExchangeDN and parses it into two parts. The first part is the site folder name. The second part is the subject of the message.

• Outlook Web Access and Outlook Mobile Access   These clients perform a DAV query for the other user, obtain the free/busy information, and then display it to the user. The DAV query is initiated from the server that hosts the Outlook Web Access or Outlook Mobile Access service (frequently the front-end server) against the user's mailbox server (back-end server), where the actual free/busy lookup occurs.

  Note

  For free/busy lookups to work, recipient information must be available in Active Directory, so that the target free/busy system folder can be determined. Therefore, you must enable directory synchronization with Lotus Notes or Novell GroupWise, if you want to synchronize free/busy information using Calendar Connector. As mentioned earlier in this section, Connector for Lotus Notes and Connector for Novell GroupWise create mail-enabled contacts with a legacyExchangeDN address that corresponds to the connector's local administrative group. Because of this dependency, Calendar Connector must be installed in the same administrative group as Connector for Lotus Notes or Connector for Novell GroupWise. You should install Calendar Connector on the same server as Connector for Lotus Notes or Connector for Novell GroupWise.

Free/Busy Publishing Agent (MadFB)

MadFB enables Outlook Web Access and Outlook Mobile Access to publish free/busy data. As a secondary function, MadFB also purges outdated free/busy data. By default, MadFB tries to maintain free/busy data on all non-front-end servers running Exchange Server each day at 2:00 A.M. MadFB on each server maintains the default public folder stores associated with each server's local mailbox stores (even if those public folder stores are located on another server). MadFB runs within the System Attendant process.

The MadFB maintenance process includes:

• Fixing the URLs of free/busy items   A free/busy item must be in "canonical" form, which means that the item must have a URL ending with a normalized subject, such as USER-/CN=RECIPIENTS/CN=TED. Items might be in non-canonical form because of duplicates. For example, one of the URLs might have a tie-breaking "-x" added, or one of the URLs might point to an item that was upgraded or replicated from Exchange Server 5.5, in which case, the URL includes a GUID. The normalized subject is determined by the last part of the legacyDN (for example, CN=RECIPIENT,CN=TED).

• Deleting duplicate free/busy messages   It is possible for Outlook to create a duplicate free/busy message. To prevent overriding existing messages, the Exchange store appends an "–X" (without the quotation marks, where x is an incremental counter for the duplicate) to the normalized subject. MadFB deletes messages that have subject lines in non-canonical form.

MadFB keeps the earliest dated message and deletes the remaining messages, which ensures deterministic replication, in which duplicate entries are always deleted. For example, if MadFB keeps the newest message and deletes the remaining messages, the conflicting message [X-2] is persisted through replication. This occurs because X on PF1 and X-2 on PF2 are first deleted, and the newer versions of X-2 on PF1 and X from PF2 are replicated. Therefore, these become the newest entries, and the cycle then is repeated.

Cleaning Free/Busy Data

There are three ways to remove unwanted free/busy data. You can use an Outlook command line startup switch, you can perform a server-side process on the server that is running Exchange Server, or you can use Outlook Web Access to delete items manually.

Outlook Startup Switch

Outlook's /cleanfreebusy command-line startup switch is used to solve meeting scheduling problems. This switch cannot help with general appointment problems, because it does not delete the free/busy item on the public folder store, but instead deletes the local free/busy message (LocalFreeBusy) generated by the Outlook client. The LocalFreeBusy message exists as a hidden item in the user's Calendar folder in the mailbox on the server. This item contains a binary large object with the user's free/busy information, information about delegates that are allowed to schedule appointments for the user, and auto-accept settings. Resource mailboxes are usually configured to accept meeting requests automatically if no conflict with an existing appointment exists. The LocalFreeBusy item is replicated to the public folder store so that all users in the Exchange organization can see your free/busy information and use it for meeting scheduling.

If delegates receive an error message when attempting to modify the manager's calendar, running /cleanfreebusy on the manager's calendar while the delegates have Outlook shut down resets particular properties on the manager's public folder store. The next time that delegates start Outlook, they retrieve the new free/busy data from the manager's LocalFreeBusy item, thereby solving most delegate errors.

This delegate meeting scheduling problem originally occurs because the delegate client (for various reasons) re-creates the free/busy message, which results in pointers pointing to the deleted message. When the manager runs Outlook /cleanfreebusy in this state, the manager's client re-creates the local free/busy message and stamps the root folders with the new entry ID, which allows everyone to access the local free/busy message again.

Cleaning Using Outlook Web Access

The free/busy messages reside in a public folder under the SCHEDULE+ FREE BUSY container in the non-ipm subtree of the public folder hierarchy. The non-ipm subtree is a hidden tree, but you can use Outlook Web Access to access this tree and open the free/busy folder of an administrative group. It is then possible to delete free/busy items manually. For example, to access the non-ipm subtree on a server that is named Server01, use the following URL: https://server01/public/non_ipm_subtree/. The SCHEDULE+ FREE BUSY container is displayed as a regular public folder. Under this container, you find the free/busy folders.

Calendar Connector Components

Because Calendar Connector does not transfer e-mail messages between Exchange and Lotus Notes or Novell GroupWise, this connector does not have a connector mailbox in the Exchange store or a proxy address generation DLL or adrType object in Active Directory. Nevertheless, Calendar Connector is a MAPI-based connector, because it relies on MAPI for Exchange store communication and on Active Directory Service Interfaces (ADSI) for communication with Active Directory.

The following table lists the important components of Calendar Connector.

Calendar Connector components

Free/Busy Lookups to and from Lotus Notes

The following figure illustrates how Calendar Connector integrates with a Lotus Notes messaging environment.

Synchronizing free/busy information with Lotus Notes

In Calendar Connector, Notescal.dll communicates with Lotus Notes and Domino through the Lotus Notes Client API to transfer requests for Lotus Notes free/busy information to the Lotus Notes Schedule Manager task. Schedule Manager is a task that runs on a Lotus Domino server, which manages a Lotus Notes database named Busytime.nsf. The Busytime.nsf database holds free/busy information for all the users on the server and for resources, such as conference rooms, identified in the server's public address book.

Free/Busy Lookups from Exchange 2003

Calendar Connector performs the following steps for free/busy lookups for Lotus Notes users from Exchange Server 2003:

1. Mapical.dll intercepts the free/busy request and checks for existing free/busy records in the free/busy system folder. If the record has been updated within the time frame specified under Maximum age in minutes of foreign free/busy data in Exchange that can be used without querying the foreign calendar in the Calendar Connector configuration, Mapical.dll returns this data immediately.

   Note

   This mechanism only works if Calendar Connector is running on the server where the free/busy folder resides. It is possible, for example, to replicate the free/busy folder to other servers in remote administrative groups, in which case the users who query these public folder instances might receive outdated information. Exchange Server 2003 only returns the information currently available in the requested free/busy messages. To avoid this problem, you must install separate Calendar Connector instances for each replica of the free/busy folder.

2. If a free/busy record does not exist or is beyond the maximum time limit, Mapical.dll passes the free/busy query to Notescal.dll to update the target user's free/busy record in the Exchange free/busy folder.

3. Notescal.dll receives the free/busy query from Mapical.dll and passes it to the Lotus Notes Schedule Manager task.

4. The Schedule Manager task retrieves the information for local users from the Busytime.nsf database. For users on downstream Lotus Domino servers, Schedule Manager communicates with the Lotus Notes Calendar Connector task that is also running on the Lotus Domino server to locate the free/busy information.

5. The Lotus Notes Calendar Connector task determines the domain of the target user and reads the Calendar Server Name field from the domain document. Calendar Connector then communicates with the remote calendar server to perform the free/busy query.

6. The Lotus Notes Calendar Connector task returns the information to the Schedule Manager task.

7. The Schedule Manager task returns the information to Notescal.dll.

8. Notescal.dll passes the information to Mapical.dll, which updates the Lotus Notes user's free/busy record in the system folder.

9. Mapical.dll returns the information to the Outlook user.

   Note

   If the non-Exchange system responds within the period of time specified under Maximum number of seconds to wait for response from foreign calendars in the Calendar Connector configuration, the data is written to the target user's free/busy record in the Exchange free/busy folder and returned to the client. If the non-Exchange system does not respond within the allowed time frame (or if Calendar Connector is not running), Exchange Server 2003 returns the existing data from the free/busy record to the client, without first updating the target user's free/busy record.

Free/Busy Lookups from Lotus Notes

Calendar Connector performs the following steps for free/busy lookups for Exchange Server 2003 users from Lotus Notes:

1. The Lotus Notes client passes the free/busy query to the Schedule Manager task.

2. The Schedule Manager task determines that the request is for a non-local user and passes it to the Calendar Connector task.

3. The Calendar Connector task reads the Person document for the Exchange user and determines that the user is in a foreign domain. The Calendar Connector task checks the Calendar System field in the Foreign Domain document for the Exchange Server 2003 organization. The Calendar System field identifies the name of the add-in program that handles the free/busy lookups on the Lotus Domino server, which is the Exchange Calendar Connector add-in (ExCalCon.exe).

4. The Calendar Connector task passes the free/busy request to ExCalCon.exe.

5. ExCalCon.exe passes the request to the Notescal.dll component, which processes the request and communicates with Mapical.dll to obtain the free/busy information for the Exchange user from the free/busy system folder.

6. Notescal.dll passes the response back to ExCalCon.exe, which in turn passes the information to the Calendar Connector task.

7. The Calendar Connector task passes the data to Schedule Manager.

8. Schedule Manager delivers the free/busy information to the Lotus Notes user.

Free/Busy Lookups to and from Novell GroupWise

As shown in the following figure, Gwisecal.dll communicates with Novell GroupWise through Connector for Novell GroupWise and Novell GroupWise API Gateway. Free/busy requests are transferred within Novell GroupWise in the form of system messages. The architecture of Connector for Novell GroupWise is discussed earlier in this section.

Synchronizing free/busy information with Novell GroupWise

Calendar Connector performs the following steps for free/busy lookups for Novell GroupWise users from Exchange Server 2003:

1. Mapical.dll intercepts the free/busy request and checks for existing free/busy records in the free/busy system folder. If the record is updated within the time frame specified under Maximum age in minutes of foreign free/busy data in Exchange that can be used without querying the foreign calendar in the Calendar Connector configuration, Mapical.dll returns this data immediately.

2. If a free/busy record does not exist for the Novell GroupWise user or exceeds the maximum time limit, Mapical.dll passes the free/busy query to Gwisecal.dll to update the target user's free/busy record in the Exchange free/busy folder.

3. Gwisecal.dll translates the request to a SEARCH-type keyword-based text file and puts it in the \Program Files\Exchsrvr\Conndata\GWRouter\Togwise directory. The message originator of this SEARCH-type message is System Attendant. The message is addressed to the Novell GroupWise user for whom Calendar Connector requests the free/busy information. The following is an example of a SEARCH-type request:

   WPC-API= 1.2;

   MSG-TYPE= Search;

   Msg-ID= AAIMIDMI:2003.12.2.21.28:2004.1.31.21.28:2003.12.3.5.28.51;

   From=

   WPD= CONTOSO_DOM;

   WPPO= Exchange Gateway;

   WPU= Microsoft System Attendant;

   CDBA= CONTOSO_DOM.Exchange Gateway.Microsoft System Attendant; ;

   To=

   WPD= CONTOSO_DOM;

   WPPO= CONTOSO_PO;

   WPU= FrankM;

   CDBA= CONTOSO_DOM.CONTOSO_PO.FrankM; ;

   Begin-Time= 2/12/2003 21:28;

   End-Time= 31/1/2004 21:28;

   -END-

4. The Router for Novell GroupWise obtains the message from the \Togwise directory and puts it in the API_IN directory of Novell GroupWise API Gateway.

5. The API gateway processes the message according to the MSG-TYPE keyword and puts it in the WPCSIN directory for the Novell GroupWise MTA.

6. The Novell GroupWise MTA routes the message to the home post office of the GroupWise user and passes it to the appropriate Novell GroupWise Post Office Agent (POA).

7. The Novell GroupWise POA processes the request and returns the resulting free/busy information to the GroupWise MTA in the form of a SEARCH message.

8. The GroupWise MTA transfers the message to the WPCSOUT directory in the API gateway directory and the API gateway transfers the message to the API_OUT directory.

9. Router for Novell GroupWise obtains the SEARCH message from the API_OUT directory and places it according to the MSG-TYPE keyword in the \Program Files\Exchsrvr\Conndata\GWRouter\freebusy directory. The following is an example of a response to a free/busy query:

   WPC-API= 1.2;

   Header-Char= T50;

   Msg-Type= SEARCH;

   Orig-Msg-ID= AAIMIDMI:2003.12.2.21.28:2004.1.31.21.28:2003.12.3.5.28.51;

   To=

   CDBA= CONTOSO_DOM.Exchange Gateway.Microsoft System Attendant;

   ;

   Busy-For=

   CDBA= CONTOSO_DOM.CONTOSO_PO.FrankM;

   Busy-Report=

   Start-Time= 11/12/03 17:00;

   End-Time= 12/12/03 8:00; ,

   Start-Time= 12/12/03 17:00;

   End-Time= 15/12/03 8:00; ,

   Start-Time= 15/12/03 17:00;

   End-Time= 16/12/03 8:00; ,

   Start-Time= 16/12/03 17:00;

   End-Time= 17/12/03 8:00; ,

   Start-Time= 17/12/03 17:00;

   End-Time= 18/12/03 8:00; ,

   Start-Time= 18/12/03 17:00;

   End-Time= 19/12/03 8:00; ,

   ;

   Send-Options= None;

   -END-

10. Gwisecal.dll retrieves the message and translates it to Exchange format. Gwisecal.dll then passes the data to Mapical.dll.

11. Mapical.dll updates the free/busy record for the Novell GroupWise user in the free/busy system folder.

12. Exchange Server 2003 returns the free/busy information to the requesting Outlook user.

Free/Busy Lookups from Novell GroupWise

Calendar Connector performs the following steps for free/busy lookups for Exchange Server 2003 users from Novell GroupWise:

1. A Novell GroupWise user performs a free/busy search for an Exchange user. The Novell GroupWise client generates a SEARCH message, which the Novell GroupWise system transfers to the API gateway.

2. The API gateway transfers the SEARCH message from the WPCSOUT directory to the API_OUT directory, where Router for Novell GroupWise picks it up and places it according to the MSG-TYPE keyword in the \Program Files\Exchsrvr\Conndata\GWRouter\FreeBusy directory. The message is addressed to the Exchange user for whom the Novell GroupWise user requests free/busy information. The structure of the message is similar to the one generated by Gwisecal.dll for queries from Exchange Server users.

3. Gwisecal.dll obtains the SEARCH message from the \FreeBusy directory, translates it into Exchange Server format, and then passes the request to Mapical.dll.

4. Mapical.dll processes the free/busy query and returns the requested information to Gwisecal.dll.

5. Gwisecal.dll translates the request to a SEARCH-type response and puts it in the \Program Files\Exchsrvr\Conndata\GWRouter\Togwise directory. The structure of the message is similar to the one generated by the Novell GroupWise system for a response to queries from Exchange users.

6. Router for Novell GroupWise obtains the message from the \Togwise directory and puts it in the API_IN directory of the API gateway.

7. The Novell GroupWise system routes the response to the user who issued the free/busy query.