Inside SharePoint Integrating Office Applications
Code download available at: SharePoint2008_07.exe (8,622 KB)
Microsoft Windows SharePoint Services (WSS) 3.0 and Microsoft Office SharePoint Server (MOSS) 2007 integrate seamlessly with desktop applications in the 2007 Microsoft Office system, facilitating collaboration on documents, spreadsheets, calendars, contact information, and more. In fact, the integration is so seamless you
can say that the 2007 Office system is a unified office solutions platform.
This is great for organizations that focus on Microsoft® Office technology to boost information worker productivity. However, organizations with a diverse portfolio of Office applications cannot enjoy the same level of interoperability out of the box. The 2007 Office system provides the necessary integration capabilities, yet the interfaces and components are minimally documented and don't work under all circumstances. This makes it hard for third-party vendors to integrate SharePoint® technology into their applications, and even harder for their customers to provide information workers with a unified experience.
In this column, I'll show you how Office applications integrate and communicate with SharePoint, and how you can integrate non-Microsoft applications with SharePoint based on the same principles. But first, I'll briefly discuss the server configuration settings, client-side components, and communication protocols used to achieve seamless integration.
Once these details are clear, I can approach the integration of apps that come without built-in SharePoint support. In this effort, I'll go beyond the usual level that typically focuses on implementing IFilter components to extend search capabilities and add custom icons on SharePoint servers. Extending search and returning results with proper icons does not achieve full app integration; users might also want to open these documents directly from within the SharePoint user interface.
This is where it gets complicated. You need a client-side component that isn't readily available if you haven't deployed Office on your workstations. But even if you have deployed Office, you might find that this component does not work reliably, depending on your SharePoint site topology and configuration.
To solve this problem, I created a custom solution that lets you integrate any application, such as Notepad, Adobe Reader, or Autodesk AutoCAD, without Office deployment requirements. This custom solution also illustrates why application integration based on standard Office components fails in some situations. You can find this solution, as well as step-by-step deployment and configuration instructions, in the companion material for this column, available in the Code Download section of .com.
Office Application Integration
Figure 1 Working with a Word document in SharePoint (Click the image for a larger view)
Despite this complexity, the user experience on a Windows workstation with the 2007 Office system installed is seamless. Working with documents in a SharePoint library is pretty much like working with local files or files on a network share.
However, on a Windows workstation without Office 2003 or the 2007 Office system, the UI is different. Clicking on New Document or Edit in Word only brings up a dialog box informing you that a Windows SharePoint Services-compatible application is not available.
On the other hand, communication between SharePoint and the database server is transparent to the application, and so are the indexing processes on the Web server to facilitate searches. For this reason, I will not discuss these aspects further in this article, but I recommend that you check out the Microsoft Filter Pack documented in the Microsoft Knowledge Base article "How to Register Microsoft Filter Pack with Windows SharePoint Services 3.0," available at support.microsoft.com/kb/946338. Now, let's focus our attention on adding commands, implementing necessary client-side components, and facilitating application communication.
Extending the SharePoint UI
There's nothing to stop you from opening the Ows.js file within a text editor (the file can be found in the COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\12\Template\Layouts\1033 folder on your SharePoint front-end servers) and changing the way the createNewDocumentWithProgIDCore, editDocumentWithProgIDNoUI, and DispDocItem functions work. But there's a better method that doesn't require you to change the WSS code base in unsupported ways: using content types and document type mappings.
In my article "Standardize Data Management with Custom Content Types" (see TechNet Magazine, February 2008, technet.microsoft.com/magazine/cc194408.aspx), I explained how to create global and site-specific content types for managing documents and other content in SharePoint document libraries and lists. You can also check out the worksheet Text Content Type.pdf in this article's companion material, which illustrates how to create a new content type for .text files and associate this content type with a document library. The result is that you can select the Text Content command from the New menu just as you can select the Document command for Word documents (see Figure 2).
Figure 2 A custom content type in the SharePoint UI (Click the image for a larger view)
The ECB menu is likewise extensible. You might have expected the ECB menu to be content-type aware, but that's not how the current WSS version implements this menu. Instead, you must register your document type in Docicon.xml, which you can find in the %COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\12\Template\Xml folder on every SharePoint front-end server.
By adding the following document type mapping to the <ByExtension> section of Docicon.xml, for example, you will cause SharePoint to display the Edit in Notepad command (see the Text Content Type.pdf file for more detailed instructions):
<Mapping Key="text" Value="ictxt.gif" EditText="Notepad" OpenControl="SharePoint.OpenDocuments"/>
SharePoint Products and Technologies Web Site
You should note that the OpenControl that's named SharePoint.OpenDocuments is referring to an ActiveX® control that ships with recent versions of Office (%PROGRAMFILES%\Microsoft Office\Office12\Owssupp.dll). If this file exists on your workstations and if the specified document icon (Value parameter) is in the %COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\12\Template\Images folder on your SharePoint servers, you might have already accomplished most of the necessary integration work.
The Windows SharePoint Services 3.0 SDK includes some information about the client-side APIs that ship with the 2007 Office system, including the OpenDocuments control. See the section "Client-Side API Reference" at msdn2.microsoft.com/ms440037.
Using the Office OpenDocuments Control
The OpenDocuments control addresses the most vital application integration needs, but it requires Office 2003 or the 2007 Office system, and its capabilities are somewhat limited. The commands on the New menu may not always work, and they sometimes display what might be misleading user notifications.
As illustrated in Figure 3, the OpenDocuments control informs the user that the required application is not correctly installed or that the template file cannot be opened, but neither is the case. The Edit in functionality also has issues. The second error message shown in the foreground of Figure 3 is frequently seen. It is my all-time favorite because it misleads even SharePoint experts, but I'll tell you more about it in a moment.
Figure 3 Misleading error messages of the OpenDocuments control (Click the image for a larger view)
Nevertheless, the OpenDocuments control is useful in environments with recent versions of Office installed on all workstations. Among other things, you can hide your content types from the New menu (display the Document Library Settings, click on Change New Button Order And Default Content Type, and then uncheck the Visible checkbox for all content types in question), so that users won't encounter the first error message. You can also keep your site topology simple in order to avoid Web-Based Distributed Authoring and Versioning (WebDAV) communication problems; this eliminates that second error message.
Communicating with SharePoint
So far, I have managed to extend the SharePoint UI and perhaps start an application by means of the OpenDocuments control, but my application still needs a way to communicate with SharePoint to access the data. As always, SharePoint supports multiple methods to do this, such as Microsoft Office FrontPage® Server Extensions, WSS remote procedure calls (RPCs), WebDAV, and Web services. In fact, Office apps, such as Word 2007, might use any or all of these communication methods, depending on how you access a document, such as through Web folders, mapped network drives, or the SharePoint UI.
The SharePoint client/server communication methods each rely on HTTP as the underlying protocol. For example, FrontPage and WSS RPCs all use HTTP POST and HTTP GET requests directed at ISAPI extensions that reside on SharePoint servers within the %COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\12\ISAPI folder and all its subfolders.
One of the most important ISAPI extensions is Owssvr.dll, which implements, among other things, functionality to work with lists and document libraries. Figure 4 shows the SharePoint Save dialog box in Word 2007 (left side) and in a browser (right side) opened directly through the URL request http://sharepoint/HR/Administration/_vti_bin/owssvr.dll?dialogview=FileSave&location=Shared%20Documents&FileDialogFilterValue=*.docx. The similarities between the two screenshots are obvious.
Figure 4 The SharePoint Save dialog box in Word 2007 and in Internet Explorer (Click the image for a larger view)
Other important ISAPI extensions are Author.dll, which implements FrontPage and WSS RPCs for client-side editing operations, such as uploading, renaming, and deleting documents; Admin.dll to manage sites and perform a number of other administrative tasks; and Shtml.dll to support HTML form submissions.
It is generally impossible to add FrontPage and WSS RPC support to existing applications, such as Notepad or Adobe Reader, without having access to source code. But you could provide the necessary communication facility by means of the Web Client feature included in Windows.
SharePoint also supports WebDAV through Httpext.dll, which resides in the %WINDIR%\System32\Inetsrv\ folder, but let's stay on the client side. Computers running Windows Server® 2008 or Windows Vista® have the Web Client feature installed by default. You can find a corresponding WebClient service in the Services applet in Administrative Tools in the Control Panel. On Windows XP and Windows Server 2003, you must install the Web Client explicitly. In any case, make sure the WebClient service is started and the Startup type is set to Automatic.
Figure 5 is a depiction of the architecture of the Web Client. The WebClient service is implemented in a user-mode DLL (Webclnt.dll) the Service Control Manager loads in a Svchost.exe host process. Webclnt.dll provides the Network Provider interface for non-I/O operations (such as authenticating the user for WebDAV access; mounting SharePoint sites as network drives; enumerating SharePoint sites, lists, and document libraries as network resources; and disconnecting mounted drives).
Figure 5 The WebDAV Client Redirector architecture (Click the image for a larger view)
To perform this work, Webclnt.dll communicates with a kernel-mode file system driver that provides the actual redirector functionality. The WebDAV Client Redirector Driver (Mrxdav.sys) is based on the Redirected Drive Buffering Subsystem (RDBSS), which integrates with I/O Manager and other kernel components in order to provide the services of a remote file system. Mrxdav.sys implements the WebDAV communication facility to support file system-level access to SharePoint sites and document libraries.
Having access to SharePoint sites and document libraries through a network redirector eliminates the need for FrontPage and WSS RPC support in user applications. You can map network drives to document libraries (such as net use x: http://wss/doclib/Shared%20Documents), and you can access SharePoint resources through UNC paths as well.
The URL http://wss/doclib/Shared%20Documents maps to \\wss\doclib\Shared%20Documents. So you have choices to open a document in an application. For example, you can open a document in Notepad with the HTTP path http://wss/doclib/Shared%20Documents/New%20Text%20Document.txt or the UNC path \\wss\doclib\Shared%20Documents\New%20Text%20Document.txt.
Unfortunately, the Web Client feature has several limitations. You cannot access custom properties or Web applications that use custom TCP ports. The Web Client that ships with Windows Vista also fails if the user does not have access to a parent site in the hierarchy, as illustrated in WebDAV Access.pdf (see the companion material).
The path http://sharepoint/HR/Administration/Shared%20Documents/ includes a nonexisting root site (that is http://sharepoint), yet without access to the root, the Web Client cannot determine the capabilities of the Web server. The Web server rejects the Web Client's OPTIONS request with Status Code 401, which states that access is unauthorized and consequently the Web Client continues to prompt for user credentials, as shown in Figure 6, even though the user has administrative access to the site collection sharepoint/HR and all sites in it.
Figure 6 Unsuccessful WebDAV access (Click the image for a larger view)
When using the OpenDocuments control, you receive the error message displayed in Figure 3 if the Web Client fails to open a document. The application is available and the document type mapping is correct. It is the document that cannot be accessed through the WebDAV redirector.
Implementing a Custom OpenControl Solution
Generally, you have two options to address the Web Client shortcomings. You can wait until Microsoft provides an updated Web Client version, or you can implement a custom OpenControl solution that can deal with the current situation. Implementing a custom OpenControl is not a trivial task, but it eliminates the need for Office on your workstations, it enables you to handle the New command in addition to the Edit in command meaningfully, and it allows you to address situations when the Web Client fails.
Figure 7 The AppStart architecture (Click the image for a larger view)
If a document is opened for editing, Biblioso.dll verifies that the file exists and starts the associated application along with the path to the document if the file is directly accessible through WebDAV. If the file is not accessible, Biblioso.dll starts an out-of-process COM server that in turn loads OpenDocsUtility.dll to download the file through HTTP and start the application along with the path to the downloaded document.
The out-of-process COM server enables the solution to break out of the Internet Explorer® process, which restricts downloads to the Temporary Internet folder in Protected Mode. Users must be able to download files without the Protected Mode restrictions, and an out-of-process COM server working as an application broker provides this option.
Developing out-of-process COM servers in .NET is not supported, so I switched to C/C++ for this executable. I only implemented the bare minimum Save As dialog box in C++. To keep the solution as straightforward as possible and to keep development overhead low, I placed the actual download code in a .NET assembly (OpenDocsUtility.dll), which I then call through another COM interface.
In order to facilitate deployment, I added a Setup project to the solution. Among other things, the setup routine registers all COM components and writes application-specific settings to HKEY_LOCAL_MACHINE\SOFTWARE\Biblioso\AppStart. The most important parameters are both AllowedApps and AllowedFileTypes. The AppStart solution will only work with those applications and file types that you must explicitly specify in these parameters.
The setup routine also creates an elevation policy for the out-of-process COM server so that Biblioso.dll in the Internet Explorer process can start AppBrokerEngine.exe without triggering security warnings. If you are interested in learning more about Internet Explorer Protected Mode and how to deal with it in application development, I highly recommend Marc Silbey and Peter Brundrett's article "Understanding and Working in Protected Mode Internet Explorer," available at msdn2.microsoft.com/bb250462.
When examining the AppStart components, keep in mind that this solution was developed simply to show what can be done; it is not yet ready for production environments. There wasn't enough time to optimize the code and test the solution thoroughly, or to document the features, apart from source code comments.
WSS 3.0 and MOSS 2007 provide extensive application integration capabilities for a seamless user experience when working with documents and other items in SharePoint document libraries and lists. The desktop applications in the 2007 Office system demonstrate this very vividly, and it is possible to achieve the same level of integration and usability with non- Office applications as well.
Another important piece of the puzzle is the Web Client, included and installed by default on Windows Vista. The Web Client implements a WebDAV redirector and remote file system driver so that any application can access resources in SharePoint lists and document libraries similar to file shares on the network. Although the Web Client that ships with Windows Vista has shortcomings, it is a vital piece of the application integration story.
WebDAV support also bridges the gap between applications running on non-Windows workstations and SharePoint. COM and ActiveX control technology is generally not available on these operating systems, so you can't use OpenControl components to start applications automatically, but most operating systems include WebDAV redirectors so that users can at least access documents directly in document libraries without having to download the files to the local workstations.
WSS 3.0 and MOSS 2007 are cornerstones of the 2007 Office system success story, and users of third-party Office-based applications can benefit from SharePoint in the same way. Integration capabilities reach far beyond Office. By taking full advantage of SharePoint, you can build a unified office solutions platform that provides the same user experience for Office and third-party software and, as a result, boosts productivity for information workers.
Pav Cherny is an IT expert and author specializing in Microsoft technologies for collaboration and unified communication. His publications include white papers, product manuals, and books with a focus on IT operations and system administration. Pav is President of Biblioso Corporation, a company that specializes in managed documentation and localization services.
© 2008 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited