Extend SharePoint to Integrate Other Office Applications

Article
09/08/2016

Inside SharePointIntegrating Office Applications

Pav Cherny

Code download available at: SharePoint2008_07.exe(8,622 KB)

Office Application Integration
Extending the SharePoint UI
Using the Office OpenDocuments Control
Communicating with SharePoint
Implementing a Custom OpenControl Solution
Summing Up

Microsoft Windows SharePoint Services (WSS) 3.0 and Microsoft Office SharePoint Server (MOSS) 2007 integrate seamlessly with desktop applications in the 2007 Microsoft Office system, facilitating collaboration on documents, spreadsheets, calendars, contact information, and more. In fact, the integration is so seamless you

can say that the 2007 Office system is a unified office solutions platform.

This is great for organizations that focus on Microsoft® Office technology to boost information worker productivity. However, organizations with a diverse portfolio of Office applications cannot enjoy the same level of interoperability out of the box. The 2007 Office system provides the necessary integration capabilities, yet the interfaces and components are minimally documented and don't work under all circumstances. This makes it hard for third-party vendors to integrate SharePoint® technology into their applications, and even harder for their customers to provide information workers with a unified experience.

In this column, I'll show you how Office applications integrate and communicate with SharePoint, and how you can integrate non-Microsoft applications with SharePoint based on the same principles. But first, I'll briefly discuss the server configuration settings, client-side components, and communication protocols used to achieve seamless integration.

Once these details are clear, I can approach the integration of apps that come without built-in SharePoint support. In this effort, I'll go beyond the usual level that typically focuses on implementing IFilter components to extend search capabilities and add custom icons on SharePoint servers. Extending search and returning results with proper icons does not achieve full app integration; users might also want to open these documents directly from within the SharePoint user interface.

This is where it gets complicated. You need a client-side component that isn't readily available if you haven't deployed Office on your workstations. But even if you have deployed Office, you might find that this component does not work reliably, depending on your SharePoint site topology and configuration.

To solve this problem, I created a custom solution that lets you integrate any application, such as Notepad, Adobe Reader, or Autodesk AutoCAD, without Office deployment requirements. This custom solution also illustrates why application integration based on standard Office components fails in some situations. You can find this solution, as well as step-by-step deployment and configuration instructions, in the companion material for this column, available in the Code Download section of .com.

Office Application Integration

From the user perspective, SharePoint menus appear to work like the Windows® Start menu. Creating a new document in a document library is simple: click on New, then on New Document, and Microsoft Office Word 2007 starts. Editing an existing document is similarly straightforward. Hover over the document, open then Edit Control Box (ECB) dropdown menu, and then click Edit in Word. It seldom comes to mind that you started Word 2007 from a Web page through JavaScript, that you are running the application locally while the document resides far away in a SQL Server® database, and that a Web server exists within the data access path, as illustrated in Figure 1.

Figure 1 Working with a Word document in SharePoint (Click the image for a larger view)

Despite this complexity, the user experience on a Windows workstation with the 2007 Office system installed is seamless. Working with documents in a SharePoint library is pretty much like working with local files or files on a network share.

However, on a Windows workstation without Office 2003 or the 2007 Office system, the UI is different. Clicking on New Document or Edit in Word only brings up a dialog box informing you that a Windows SharePoint Services-compatible application is not available.

This isn't really a surprise. There are several elements that must work together to provide the unified experience that users of Office enjoy. SharePoint must be aware of the content types maintained in the document library to render the desired commands in the New and ECB menus. In response to mouse clicks on these commands, JavaScript code must run to start the associated application, passing along the path to the document. This part is not independent of the workstation configuration because the JavaScript code runs locally. Furthermore, the associated application must communicate with SharePoint to read and possibly write files. These are the elements that you must bring together if you want to integrate your applications with SharePoint.

On the other hand, communication between SharePoint and the database server is transparent to the application, and so are the indexing processes on the Web server to facilitate searches. For this reason, I will not discuss these aspects further in this article, but I recommend that you check out the Microsoft Filter Pack documented in the Microsoft Knowledge Base article "How to Register Microsoft Filter Pack with Windows SharePoint Services 3.0," available at support.microsoft.com/kb/946338. Now, let's focus our attention on adding commands, implementing necessary client-side components, and facilitating application communication.

Extending the SharePoint UI

An abundance of options are available to extend the SharePoint user interface and functionality. You can change the design of your sites, customize ASP.NET pages, develop Web Parts, or change the JavaScript code included in WSS and MOSS to start your applications directly.

There's nothing to stop you from opening the Ows.js file within a text editor (the file can be found in the COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\12\Template\Layouts\1033 folder on your SharePoint front-end servers) and changing the way the createNewDocumentWithProgIDCore, editDocumentWithProgIDNoUI, and DispDocItem functions work. But there's a better method that doesn't require you to change the WSS code base in unsupported ways: using content types and document type mappings.

In my article "Standardize Data Management with Custom Content Types" (see TechNet Magazine, February 2008, technet.microsoft.com/magazine/cc194408.aspx), I explained how to create global and site-specific content types for managing documents and other content in SharePoint document libraries and lists. You can also check out the worksheet Text Content Type.pdf in this article's companion material, which illustrates how to create a new content type for .text files and associate this content type with a document library. The result is that you can select the Text Content command from the New menu just as you can select the Document command for Word documents (see Figure 2).

Figure 2 A custom content type in the SharePoint UI (Click the image for a larger view)

The ECB menu is likewise extensible. You might have expected the ECB menu to be content-type aware, but that's not how the current WSS version implements this menu. Instead, you must register your document type in Docicon.xml, which you can find in the %COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\12\Template\Xml folder on every SharePoint front-end server.

By adding the following document type mapping to the <ByExtension> section of Docicon.xml, for example, you will cause SharePoint to display the Edit in Notepad command (see the Text Content Type.pdf file for more detailed instructions):

<Mapping Key="text" Value="ictxt.gif" 
  EditText="Notepad"
  OpenControl="SharePoint.OpenDocuments"/>

The Key parameter identifies the file name extension, the Value parameter defines the document icon to display in the user interface, the EditText parameter defines the string that SharePoint appends to the "Edit in" command, and the OpenControl parameter specifies the ProgID of a client-side COM component. This is the ProgID that the SharePoint JavaScript functions pass to an ActiveXObject call (see Ows.js for details) to instantiate the COM object, which can be the app itself or a helper control that starts the app based on the associated file type.

SharePoint Products and Technologies Web Site

You should note that the OpenControl that's named SharePoint.OpenDocuments is referring to an ActiveX® control that ships with recent versions of Office (%PROGRAMFILES%\Microsoft Office\Office12\Owssupp.dll). If this file exists on your workstations and if the specified document icon (Value parameter) is in the %COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\12\Template\Images folder on your SharePoint servers, you might have already accomplished most of the necessary integration work.

The Windows SharePoint Services 3.0 SDK includes some information about the client-side APIs that ship with the 2007 Office system, including the OpenDocuments control. See the section "Client-Side API Reference" at msdn2.microsoft.com/ms440037.

Using the Office OpenDocuments Control

The OpenDocuments control addresses the most vital application integration needs, but it requires Office 2003 or the 2007 Office system, and its capabilities are somewhat limited. The commands on the New menu may not always work, and they sometimes display what might be misleading user notifications.

As illustrated in Figure 3, the OpenDocuments control informs the user that the required application is not correctly installed or that the template file cannot be opened, but neither is the case. The Edit in functionality also has issues. The second error message shown in the foreground of Figure 3 is frequently seen. It is my all-time favorite because it misleads even SharePoint experts, but I'll tell you more about it in a moment.

Figure 3 Misleading error messages of the OpenDocuments control (Click the image for a larger view)

Nevertheless, the OpenDocuments control is useful in environments with recent versions of Office installed on all workstations. Among other things, you can hide your content types from the New menu (display the Document Library Settings, click on Change New Button Order And Default Content Type, and then uncheck the Visible checkbox for all content types in question), so that users won't encounter the first error message. You can also keep your site topology simple in order to avoid Web-Based Distributed Authoring and Versioning (WebDAV) communication problems; this eliminates that second error message.

Communicating with SharePoint

So far, I have managed to extend the SharePoint UI and perhaps start an application by means of the OpenDocuments control, but my application still needs a way to communicate with SharePoint to access the data. As always, SharePoint supports multiple methods to do this, such as Microsoft Office FrontPage® Server Extensions, WSS remote procedure calls (RPCs), WebDAV, and Web services. In fact, Office apps, such as Word 2007, might use any or all of these communication methods, depending on how you access a document, such as through Web folders, mapped network drives, or the SharePoint UI.

The SharePoint client/server communication methods each rely on HTTP as the underlying protocol. For example, FrontPage and WSS RPCs all use HTTP POST and HTTP GET requests directed at ISAPI extensions that reside on SharePoint servers within the %COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\12\ISAPI folder and all its subfolders.

One of the most important ISAPI extensions is Owssvr.dll, which implements, among other things, functionality to work with lists and document libraries. Figure 4 shows the SharePoint Save dialog box in Word 2007 (left side) and in a browser (right side) opened directly through the URL request https://sharepoint/HR/Administration/_vti_bin/owssvr.dll?dialogview=FileSave&location=Shared%20Documents&FileDialogFilterValue=*.docx. The similarities between the two screenshots are obvious.

Figure 4 The SharePoint Save dialog box in Word 2007 and in Internet Explorer (Click the image for a larger view)

Other important ISAPI extensions are Author.dll, which implements FrontPage and WSS RPCs for client-side editing operations, such as uploading, renaming, and deleting documents; Admin.dll to manage sites and perform a number of other administrative tasks; and Shtml.dll to support HTML form submissions.

It is generally impossible to add FrontPage and WSS RPC support to existing applications, such as Notepad or Adobe Reader, without having access to source code. But you could provide the necessary communication facility by means of the Web Client feature included in Windows.

SharePoint also supports WebDAV through Httpext.dll, which resides in the %WINDIR%\System32\Inetsrv\ folder, but let's stay on the client side. Computers running Windows Server® 2008 or Windows Vista® have the Web Client feature installed by default. You can find a corresponding WebClient service in the Services applet in Administrative Tools in the Control Panel. On Windows XP and Windows Server 2003, you must install the Web Client explicitly. In any case, make sure the WebClient service is started and the Startup type is set to Automatic.

Figure 5 is a depiction of the architecture of the Web Client. The WebClient service is implemented in a user-mode DLL (Webclnt.dll) the Service Control Manager loads in a Svchost.exe host process. Webclnt.dll provides the Network Provider interface for non-I/O operations (such as authenticating the user for WebDAV access; mounting SharePoint sites as network drives; enumerating SharePoint sites, lists, and document libraries as network resources; and disconnecting mounted drives).

Figure 5 The WebDAV Client Redirector architecture (Click the image for a larger view)

To perform this work, Webclnt.dll communicates with a kernel-mode file system driver that provides the actual redirector functionality. The WebDAV Client Redirector Driver (Mrxdav.sys) is based on the Redirected Drive Buffering Subsystem (RDBSS), which integrates with I/O Manager and other kernel components in order to provide the services of a remote file system. Mrxdav.sys implements the WebDAV communication facility to support file system-level access to SharePoint sites and document libraries.

Having access to SharePoint sites and document libraries through a network redirector eliminates the need for FrontPage and WSS RPC support in user applications. You can map network drives to document libraries (such as net use x: https://wss/doclib/Shared%20Documents), and you can access SharePoint resources through UNC paths as well.

The URL https://wss/doclib/Shared%20Documents maps to \\wss\doclib\Shared%20Documents. So you have choices to open a document in an application. For example, you can open a document in Notepad with the HTTP path https://wss/doclib/Shared%20Documents/New%20Text%20Document.txt or the UNC path \\wss\doclib\Shared%20Documents\New%20Text%20Document.txt.

Unfortunately, the Web Client feature has several limitations. You cannot access custom properties or Web applications that use custom TCP ports. The Web Client that ships with Windows Vista also fails if the user does not have access to a parent site in the hierarchy, as illustrated in WebDAV Access.pdf (see the companion material).

The path https://sharepoint/HR/Administration/Shared%20Documents/ includes a nonexisting root site (that is https://sharepoint), yet without access to the root, the Web Client cannot determine the capabilities of the Web server. The Web server rejects the Web Client's OPTIONS request with Status Code 401, which states that access is unauthorized and consequently the Web Client continues to prompt for user credentials, as shown in Figure 6, even though the user has administrative access to the site collection sharepoint/HR and all sites in it.

Figure 6 Unsuccessful WebDAV access (Click the image for a larger view)

When using the OpenDocuments control, you receive the error message displayed in Figure 3 if the Web Client fails to open a document. The application is available and the document type mapping is correct. It is the document that cannot be accessed through the WebDAV redirector.

Implementing a Custom OpenControl Solution

Generally, you have two options to address the Web Client shortcomings. You can wait until Microsoft provides an updated Web Client version, or you can implement a custom OpenControl solution that can deal with the current situation. Implementing a custom OpenControl is not a trivial task, but it eliminates the need for Office on your workstations, it enables you to handle the New command in addition to the Edit in command meaningfully, and it allows you to address situations when the Web Client fails.

If any of these issues are compelling to you, take a look at the AppStart source code included in the companion material. It shows you how to expose the OpenControl COM interfaces in a Microsoft .NET Framework assembly, which the SharePoint JavaScript code can call. The AppStart source code also demonstrates one possible way to check file accessibility and download a file to the local computer via HTTP if direct access through WebDAV is not possible. Finally, the AppStart source code responds to the New command by downloading the template associated with the content type to the local computer so that the user can start working on the document. The worksheets Text Content Type.pdf and Adobe Reader Support.pdf outline how to deploy this OpenControl solution.

The diagram shown in Figure 7 shows the AppStart architecture. My custom OpenControl component (called Biblioso.dll) exposes two identical COM interfaces that SharePoint JavaScript calls to create new documents or to open existing documents for editing (Biblioso.AppStart.2 and Biblioso.AppStart.3).

Figure 7 The AppStart architecture (Click the image for a larger view)

If a document is opened for editing, Biblioso.dll verifies that the file exists and starts the associated application along with the path to the document if the file is directly accessible through WebDAV. If the file is not accessible, Biblioso.dll starts an out-of-process COM server that in turn loads OpenDocsUtility.dll to download the file through HTTP and start the application along with the path to the downloaded document.

The out-of-process COM server enables the solution to break out of the Internet Explorer® process, which restricts downloads to the Temporary Internet folder in Protected Mode. Users must be able to download files without the Protected Mode restrictions, and an out-of-process COM server working as an application broker provides this option.

Developing out-of-process COM servers in .NET is not supported, so I switched to C/C++ for this executable. I only implemented the bare minimum Save As dialog box in C++. To keep the solution as straightforward as possible and to keep development overhead low, I placed the actual download code in a .NET assembly (OpenDocsUtility.dll), which I then call through another COM interface.

In order to facilitate deployment, I added a Setup project to the solution. Among other things, the setup routine registers all COM components and writes application-specific settings to HKEY_LOCAL_MACHINE\SOFTWARE\Biblioso\AppStart. The most important parameters are both AllowedApps and AllowedFileTypes. The AppStart solution will only work with those applications and file types that you must explicitly specify in these parameters.

The setup routine also creates an elevation policy for the out-of-process COM server so that Biblioso.dll in the Internet Explorer process can start AppBrokerEngine.exe without triggering security warnings. If you are interested in learning more about Internet Explorer Protected Mode and how to deal with it in application development, I highly recommend Marc Silbey and Peter Brundrett's article "Understanding and Working in Protected Mode Internet Explorer," available at msdn2.microsoft.com/bb250462.

When examining the AppStart components, keep in mind that this solution was developed simply to show what can be done; it is not yet ready for production environments. There wasn't enough time to optimize the code and test the solution thoroughly, or to document the features, apart from source code comments.

You should use this solution at your own risk. If you are interested in studying the source code to create your own solution, start with AppStart.cs in the Biblioso code project. This file implements the OpenControl COM interface and the entry points for the JavaScript calls from Ows.js.

Summing Up

WSS 3.0 and MOSS 2007 provide extensive application integration capabilities for a seamless user experience when working with documents and other items in SharePoint document libraries and lists. The desktop applications in the 2007 Office system demonstrate this very vividly, and it is possible to achieve the same level of integration and usability with non- Office applications as well.

At the core of the application integration architecture are COM components, which the SharePoint JavaScript functions use, along with the document path, to start the application. The 2007 Office system provides several of these COM components, optimized for specific Office application needs, though it is also possible to reuse the Office OpenDocuments control to integrate non-Microsoft applications. The OpenDocuments control meets the most basic needs. For more advanced application integration requirements, you can implement a custom control.

Fully integrating an application with SharePoint entails not only installing IFilter components and document icons to extend search and display capabilities but also involves creating custom content types and document type mappings on the SharePoint servers to provide appropriate New and Edit in commands in the SharePoint user interface. These commands call JavaScript functions that invoke methods exposed through an OpenControl component. The OpenControl component must also be available on the workstation.

Another important piece of the puzzle is the Web Client, included and installed by default on Windows Vista. The Web Client implements a WebDAV redirector and remote file system driver so that any application can access resources in SharePoint lists and document libraries similar to file shares on the network. Although the Web Client that ships with Windows Vista has shortcomings, it is a vital piece of the application integration story.

WebDAV support also bridges the gap between applications running on non-Windows workstations and SharePoint. COM and ActiveX control technology is generally not available on these operating systems, so you can't use OpenControl components to start applications automatically, but most operating systems include WebDAV redirectors so that users can at least access documents directly in document libraries without having to download the files to the local workstations.

WSS 3.0 and MOSS 2007 are cornerstones of the 2007 Office system success story, and users of third-party Office-based applications can benefit from SharePoint in the same way. Integration capabilities reach far beyond Office. By taking full advantage of SharePoint, you can build a unified office solutions platform that provides the same user experience for Office and third-party software and, as a result, boosts productivity for information workers.

Pav Cherny is an IT expert and author specializing in Microsoft technologies for collaboration and unified communication. His publications include white papers, product manuals, and books with a focus on IT operations and system administration. Pav is President of Biblioso Corporation, a company that specializes in managed documentation and localization services.

Contents

Additional resources