Chapter 1 - Learning The 5 Principles of Cooperative Network Application Development

Article
02/20/2014

Archived content. No warranty is made as to technical accuracy. Content may contain URLs that were valid when originally published, but now link to sites or pages that no longer exist.

From the book Designing Distributed Applications With XML, ASP, IE5, LDAP and MSMQ by Stephen Mohr. (ISBN: 1861002270). Copyright ©1999 by Wrox Press, Inc. Reprinted by permission from the publisher. For more information, go to https://www.wrox.com.

We often look back at events with the benefit of hindsight, wishing that we had known what was going to happen before we embarked on a task. The world of computing is evolving at such a rapid pace, that if we want to be developing applications that will survive in the workplace (perhaps so that we can build on them rather than re-building them) — and hopefully get ourselves more business in the process — we can learn a lot by attempting to look into the future.

Of course, if I had a crystal ball that could really tell the future, I would have already earned my millions and could be sailing around the world. The next best thing is to look at current research efforts that are leading the field so we can learn how others see the future shaping up. In this first chapter we will be looking at three initiatives for future distributed systems. We'll look for common features in the hopes of identifying goals for our own, present-day applications. We'll examine 3 systems:

Microsoft Windows DNA
Microsoft Millenium
Sun Jini

Although we'll have to learn enough about each technology to be able to make intelligent assessments, the object of this exercise is to extract the important enabling concepts rather than to become experts on any of these systems.

Having seen the enabling concepts that make each system so special, we will see how the principles on which they are based can be applied within current boundaries. Our findings will form the foundation for the rest of the book. By looking at the aims of distributed computing tomorrow, we can address the principles of building distributed applications today. So, what are these visions?

Looking at Future Visions

Our desire is to direct our intranet development in such a way that the resulting network actually promotes further development. Each application should draw from existing resources while adding its own resources to the network. No substantial application should be a dead end. Enhancements to applications should be organic, adding something to what came before without destroying it. This is the principal benefit of cooperative network applications. However, we need some specific goals and principles to guide us.

We can look to some future systems for inspiration. They won't provide the complete solution as they are focused on operating system enhancements and are therefore proprietary. Two of the systems we will look at are research efforts. They do, however, aim to provide powerful support for distributed computing. First, we shall look at Microsoft Windows DNA. This is an emerging effort to integrate and improve a variety of component-based technologies on the Windows platform. Next, we will look deeply into the future with Microsoft's distributed computing research project Millennium. Finally, we will hit the middle ground as we look at Sun Microsystem's Jini, the prototype distributed computing environment for Java development.

Remember that this is not simply an excursion into the realm of technical prediction. All of the systems build on the existing state of the art in distributed computing. Each represents the best efforts of top-notch talent to drive the art toward a better future. Each, to some greater or lesser extent, will influence current and near-term practice. Windows DNA, for example, is rapidly coming to fruition. It is the future of the most widely deployed desktop operating system. Meanwhile, Jini is enjoying a high level of exposure for its promise to integrate disparate devices on a common network. Although we cannot realize the full potential of these efforts — which is why they are systems for a future day — we can glean useful insight into the best practices in distributed computing. We will learn concepts that we can apply in smaller measure today, improving our applications in the process. In adopting these practices, we will, by default, align our present-day systems with future platforms and allow our systems to grow. Once we finish our examination of the future, we will return to the problems of the practical world and see what we can distill from our trip.

Windows DNA

Windows DNA — Distributed interNet Applications Architecture — draws on existing operating system technologies and adds enhancements designed to aid the development of professional-grade component-based applications. Many of the technologies embraced by the term pre-date the DNA program. The program is really an attempt to unify related efforts and better communicate Microsoft's vision for the Windows platform. DNA seeks to bring together the best features of Web-centric computing and personal computers. It aims to do this by building a platform for robust, PC-centric network computing. Of necessity, it extends the Windows platform in two directions:

Toward the communications-based model of the Web, which we refer to as Web Metaphor Computing
Deeper into network and operating system services

Pieces of DNA

Some of the pieces of DNA are present in the current Windows platform. Others are due in Windows 2000 Server (the new name for Windows NT5) or later. All are part of, or closely related to, the Windows operating system. Some are evolutionary improvements to core operating system services — networking, security, file and print services, for example — while others are application-oriented services like encryption.

Application-oriented services are not essential to the core operating system, but dramatically improve the common features available to programs written for the platform. They provide a rich infrastructure on which developers can build large-scale, robust network applications.

Web Metaphor Computing	Availability and Comments
Dynamic HTML	Available since IE 4.0
Scripting	Currently available
Familiar Web-based user interface	Available since Windows 95/Windows NT 4.0
HTTP servers and clients	Currently available
Component Software
COM components, COM+	Currently available; tool support for COM+ is forthcoming
Network Services
Transactions	Available now; will be more closely tied to components when language enhancements for COM+ become available
Queued messaging communications	Available now; coming to components with COM+
Security	Upgraded to Kerberos with Windows 2000
Directory services	Coming with Windows 2000
Universal data storage and database access	Currently available and evolving
Central network administration	Evolving

Web Metaphor Computing

Windows DNA envisions a Web-based interface as its dominant user interface and Web-style communications as a major link between application-level components. To this end, Microsoft continues to integrate its Web browser technology with the operating system. On the client, Internet Explorer is the core browsing and rendering engine. While on the server, Windows NT comes bundled with Internet Information Server (IIS). IIS provides some valuable extensions to the basic HTTP server model such as in-process scripted extensions and session state maintenance to facilitate the construction of Web-based applications that are implemented by drawing on the features provided by the Windows platform.

If Web-based techniques are to be a core paradigm for building applications, both browsers and servers must provide advanced features and robust performance. Microsoft's version of dynamic HTML (DHTML) adheres closely to the W3C document object model (DOM). Introduced with version 4.0 of their Web browser, DHTML provides powerful layout and presentation features. In addition, its event model allows Web pages to be highly interactive. DHTML permits a level of interactivity such as is seen in Java applets with performance typical of in-page scripts. Future versions of the Web browsing client will improve the performance of interface elements to achieve results closer to that experienced with applications written with compiled languages.

We will also make use of scriptlets in subsequent chapters as we develop some utilities for Web development. Scriptlets are reusable chunks of DHTML functionality formed by combining browser-based scripting with the COM component software technology provided by Windows.

The introduction of PCs led to a shift in power and control away from centralized mainframe applications. Each desktop computer hosted applications rich in features and interactivity. While this was satisfying to end users — who could now control the configuration and availability of the computing services available to them — it made central management and support of applications difficult and expensive. Web-based applications strive to combine the best features of the two models and strike a balance between them. The portion of the application residing on the user's local computer is comparatively thin, containing only that code needed to present results and an interactive interface to the user. Feature rich, fast HTML browsers permit applications to offer a thin client implementation of rich user interfaces tied to powerful server-based applications. The chief advantage of this is that programmers can rapidly produce new user interfaces to existing services using DHTML and script code. Windows DNA not only provides a rich client browser, but also integrates the browser into the core operating system to ensure the availability of these user interface services.

Component Software

Windows' component software technology is the Component Object Model (COM). Available for several years, COM provides binary level reuse of object-based software components. COM is officially neutral with regard to the choice of a programming language for implementing COM components. Many languages have been used to build COM components, including C++, Java, Visual Basic, SmallTalk and COBOL. Microsoft is promoting COM by releasing new operating system services as COM components and building new applications, notably Internet Explorer, from COM components. Access to databases and the forthcoming directory service, for example, is through COM object frameworks. Beginning with Internet Explorer 4.0, Web page scripts may be exposed as COM components for reuse in other browser-based applications. The scripting technology supported by Windows is also evolving so that COM components may be written in script, and be available to any application on the same basis as traditional COM components. We will see — and build — examples of this later in this book.

Why should Web developers care about component software? Component software technologies, especially COM and CORBA, have a reputation for being the province of advanced C++ programmers. Sometimes they are regarded as being too hard and expensive for practical use. One of the benefits of Web applications is the ability to rapidly prototype (and sometimes even deploy) a new application. The techniques are extremely flexible. Scripting languages are relatively easy to learn, and script code is easily changed. There is no compiling, so, if it doesn't work, we can change it and try again. Browsers and HTML offer powerful user interface features; try changing a dialog box written in a compiled language, then change an HTML form and see if it isn't faster. Components supercharge Web page development. Regardless of how hard they are to develop, they are no harder to use from script than the elements of a browser's document object model. On the server, components offer pre-built chunks of functionality to script, and you can buy many useful components from a thriving market of third-party vendors. Most of these components are implemented in compiled languages, so you get the features and good runtime performance.

Equally important is the fact that the vendors of component technologies are working to simplify the task of building new components. COM was originally the exclusive domain of C++ developers. Many components are now written in Visual Basic, and Microsoft is even testing technology to allow script programmers to build COM components.

As already mentioned, you are going to see numerous examples of such components throughout this book, and all the components in our toolkit are written exclusively in JavaScript and VBScript. If you have a useful bit of code, you can easily and quickly package it as a component. This is an excellent way to rapidly prototype components!

Windows DNA continues the evolution of COM with a body of enhancements collectively known as COM+. The enhancements deal chiefly with performance, integration with operating system services, and simplified implementation through implementation language support.

Performance Enhancements in COM+

The runtime environment provided by the Windows 2000 operating system has been modified to provide faster access to components, role-based security, and load balancing. The registry of COM components and their implementing software is also cached in memory, which greatly speeds the process of locating software and instantiating a component instance. For a long time programmers have asked for the ability to make asynchronous calls on COM components, contending that synchronous calls are not scalable. This is largely because a synchronous call on a COM component may time out if too many clients are making method invocations against a single remote component. The result is a failure from the point of view of the calling client. Applications relying on synchronous calls are unduly delayed when making calls against a heavily loaded server, so COM+ introduces the ability to make queued, asynchronous calls to COM components.

Integration with Operating System Services

COM+ is well integrated with operating system services, such as data access and transaction processing. COM+ also introduces a new concept called interception. Interception permits the dynamic redirection of processing in response to events and method calls. A component programmer indicates that some particular operating system or custom service should be used by his application in conjunction with particular method calls or types of operations. At runtime, calls to that component are intercepted by the operating system and redirected to the appropriate service. For example, a component might use interceptors to provide transparent access to transaction services provided by the operating system. It might also intercept instance creation calls and implement a load-balancing scheme for scalability. Using interceptors, the properties of a component may even be dynamically bound to the fields of a database table. Windows NT has added several services to the basic operating system in the last few years, and COM+ allows component programmers easy access to them through interception.

Simplified Implementation

While interceptors can be used to offer increased functionality, they can also simplify the basic task of programming COM components. Default implementations of basic COM tasks are provided by the COM+ runtime. A standard implementation for firing events is provided, for example. Prior to COM+, a programmer had to write this code himself, or make reference to some library implementation of these services. COM+ uses a declarative programming model for this. Keywords are added to component declarations as attributes, and compilers that understand COM+ provide stubs invoking the appropriate service for the attribute at runtime. Taking the example of firing an event, the programmer indicates that a COM+ component fires events from a particular interface through a keyword declaration in his code. The proper stubs to maintain lists of components wishing to receive those events are added by the compiler. When he wishes to fire a particular event in the body of the component implementation, the programmer simply invokes the method he wishes to fire. The runtime intercepts the call and walks the list, ensuring listeners receive the event.

Next I'm going to present several code fragments in order to illustrate the benefits of COM+ as compared to current COM practice. I will alternate between COM+ and the equivalent COM code. These are necessarily code fragments and must not be taken for complete implementations. In particular, I will concentrate on the declarations, leaving the actual implementing code aside. My emphasis here is on showing Web developers — many of whom will not be COM and C++ programmers — the coming changes in Windows and COM+, and how component development is made easier as part of our look into the future.

Consider a hypothetical plant manufacturing application. We have tanks holding some volume of fluid. During the manufacturing process, we charge the tank with some amount of fluid and later drain the tank of some amount of fluid. The charging process can incur faults due to leaks, overfilling, or a blocked transfer line. We can model this in COM and COM+ as a tank component that sources (fires) events to some component monitoring the process. Here is the COM+ declaration for the tank and fault event interfaces:

cointerface ITank

{
float ChargeTank([in]int nSourceTankID, [in]float fVolume);
}
cointerface IChargingFaultEvent

{
void Leak(String strFaultText);
void OverCharge(String strFaultText);
void LineBlockage(String strFaulttext);
}

The keyword cointerface specifies that these are COM interfaces.

Here's the C++ declaration for a COM+ component representing a tank full of hydrazine:

coclass HydrazineTank: implements ITank, fires IChargingFaultEvent

{
// indicate transactions required, state model, source of data, thread model
attributes:
transaction = "required";
state = "stateless";
data_source = "DSN=FluidTankDatabase";
threading = "single";
public:
Pump m_pTransferPump;
DataSource m_dsTank;
// indicates that the value of m_fVolume comes from a column in the
// db referenced by the DSN named in m_dsTank
[source=m_dsTank, column="cur_volume"] float m_fVolume;
float ChargeTank(int nSourceSerial,
float fTransferAmount) throws SQLException;
};

The keyword coclass indicates that this class will be the implementation of a COM component. This causes the compiler to provide some basic code required by all COM/COM+ components. We've indicated that it implements the ITank interface and fires events from the IChargingFaultEvent interface.

The attributes section tells the operating system several important pieces of information. First, our component requires a transaction. In COM, this cannot be done in code; instead, we have to manually perform some configuration in Microsoft Transaction Server (MTS) and add lines of code delineating the boundaries of the transaction. Next, we've indicated this is a stateless object — it does not guarantee persistent state between method invocations. This should improve scalability because resources consumed by the component may be released between method calls, but it requires us to store the volume of the tank in the database rather than in a member variable. Following this, we've indicated the source of data for the volume of the tank represented by this component. The value of m_dsTank is a DSN that refers to the database in which the volume is stored. The runtime may fetch the value of the cur_volume column and cache it in memory in our COM+ implementation. This value can be selectively refreshed from the database. In COM, we would need to do this retrieval in code every time we needed to get the value. Finally, we've indicated the threading model. The features declared by these attributes are implemented through interception. The attribute keywords tell the compiler to attach the proper stubs to our component. At runtime, these stubs intercept method calls and property references and perform the necessary tasks.

Now contrast this with a similar component declaration using present-day COM. We will use C++ to fully contrast the COM+ implementation with a COM implementation in the same implementation language. Admittedly, a Visual Basic or Java implementation would be simpler. Using the ActiveX Template Library and Microsoft Visual Studio version 6.0, the work done by the first line of the COM+ declaration — declaring this to be a COM component and indicating the interfaces used — requires the following:

class ATL_NO_VTABLE CHydrazineTank :
public CComObjectRootEx<CComSingleThreadModel>,
public CComCoClass<CHydrazineTank, &CLSID_HydrazineTank>,
public IDispatchImpl<ITank, &IID_ITank, &LIBID_SAMPLETANKLib>
{
...
BEGIN_COM_MAP(CHydrazineTank)
COM_INTERFACE_ENTRY(ITank)
COM_INTERFACE_ENTRY(IDispatch)
...
END_COM_MAP()
...

In addition to declaring this to be a COM object exposing the ITank interface, we also managed to squeeze in a declaration of the component's threading model. Clearly, though, the COM+ declaration is more readable and efficient from the programmer's standpoint.

Returning to the COM+ declaration, the member variable m_TransferPump is a pointer to a COM object representing the pump that transfers fluid in and out of our tank. When we wish to create an instance of this component for use with our tank component, we simply use the following lines in COM+:

m_pTransferPump = new Pump();
if (m_pTransferPump == NULL)
... // some error handling code

In COM, assuming the COM libraries are initialized, we require:

m_pTransferPump = NULL;

HRESULT hr = ::CoCreateInstance(CLSID_PUMP, NULL, CLSCTX_INPROC_SERVER,
IID_IPumpInterface, (void **)&m_pTransferPump);
if (SUCCEEDED(hr))
{
... // okay to use
}

When our tank object detects a fault, we want to fire an event. In COM+, that's as simple as:

OverCharge(strFaultText);

The COM+ runtime handles the task of alerting all the monitoring objects that have connected to our tank component for monitoring purposes. In COM, we have to do the work ourselves:

IEnumConnections *pEnum;
CONNECTDATA cd;

If (FAILED(m_rgpConnectionPt[0]->EnumConnections(&pEnum)))
return FALSE;

While (NOERROR==pEnum->Next(1, &cd, NULL))
{
IChargingFaultEvent *pEvt;
if (SUCCEEDED(cd.pUnk->QueryInterface(IID_IChargingFault, (PPVOID)&pEvt)))
{
pEvt->OverCharge(strFaultText);
pEvt->Release();
}
cd.pUnk->Release();
}
pEnum->Release();

COM+ offers features using the transaction and state attributes, which require much more work to achieve in COM. First, programming a stateless component is as much philosophy as declaration, whether we use COM or COM+. We have to be sure we don't rely on state between method calls. COM+ provides some default implementations for tasks that a COM programmer using MTS (Microsoft Transaction Server) must provide himself. MTS will assist the programmer by notifying the program when the runtime environment is going to take action that will disrupt state, provided the COM component supports the IObjectControl interface. This adds an entry to our interface map and three methods, declared as follows:

BEGIN_COM_MAP(CHydrazineTank)
...
COM_INTERFACE_ENTRY(IObjectControl)
...
END_COM_MAP()

public:
STDMETHOD(Activate)();
STDMETHOD_(BOOL, CanBePooled)();
STDMETHOD_(void, Deactivate)();

CComPtr<IObjectContext> m_spObjectContext;

The member variable m_spObjectContext is a pointer to an interface supported by the MTS runtime environment that allows the object to signal when it is ready to release state. This allows the runtime to reclaim resources. When the component has finished some section of work and wishes to indicate that it no longer needs to hold state, it would make the following call:

m_spObjectContext->SetComplete();

If it could not complete and wished to give up state, it would call the SetAbort() method of the IObjectContext interface.

To ensure that the component always executes within a transaction, the administrator installing the component in a particular MTS environment must configure the component's transaction attribute in the MTS runtime for Requires a Transaction or Requires a New Transaction, thereby calling for automatic transaction control.

Hopefully, these comparisons of code fragments have convinced you that COM+ simplifies life for the component creator. By giving the compiler knowledge of many of the available system services, COM+ frees the programmer to concentrate on the semantics of his application. The programmer only has to declare what services should be involved with his component and the compiler becomes responsible for adding the integration stubs needed to make this happen. Additionally, COM+ allows enforcement of the programmer's transaction requirements, thereby ensuring consistency and reducing the system administrator's burden.

Network Services

A modern operating system offers key services essential to computing, such as file and print services. Applications written to the platform tend to have some common requirements, and implementations answering these needs tend to be added to the operating system over time. These services, such as networking support and security features, are needed by many applications. Although arguably not part of the core operating system, they are so useful that programmers come to depend on them. The popularity of UNIX and Windows stem in part from the rich application services that are bundled with the core operating system. Any operating system that aspires to run mission critical applications in an inherently networked environment requires some key services which are not ordinarily found in a stand-alone environment. Windows DNA builds on existing services and adds new ones in an effort to prepare Windows NT for mission critical enterprise computing tasks. Let's take a look at some of these:

Feature	Status in Windows NT 4.0	Status in Windows 2000 Server
Security	Proprietary method	Ability to use various methods; Kerberos is used by default
Network Administration	Microsoft Management Console offers a basic framework for adding management components	Features for replicating a standard configuration to networked machines from a server added and management generally improved
Directory services	Proprietary, local registry performs similar function	Active Directory introduces a true directory service oriented to the needs of this operating system
Transaction Processing	Microsoft Transaction Server (MTS) and Distributed Transaction Coordinator offer this to COM components	MTS features merged with COM+
Messaging Middleware	Microsoft Message Queue (MSMQ), now bundled with NT Server, previously available with the NT4 Option Pack	MSMQ used by COM+ to offer asynchronous method invocations on components
Universal Data Access	COM object frameworks available; support varies by database vendor	Continues; designated as the core infrastructure for data access in Windows 2000

Security

The security services built into versions of Windows NT prior to Windows 2000 Server were proprietary and faulted by many for weakness and a lack of scalability. Windows 2000 Server introduces an implementation of the Kerberos network security protocol as its default security service. Not only does this strengthen security, it also increases interoperability with other operating systems. Implementations of Kerberos, which originated at the Massachusetts Institute of Technology, are widely available for many common operating systems.

Central Network Administration

The move to pervasive networking in commercial settings is behind the increasing demand for centralized administration of network resources. We won't go into too much detail here — one of the assumptions we make is that application developers will have little or no formal authority to make 'top-down' changes in the computing environment.

The Active Directory (covered next) will give Windows 2000 Server a central store for configuration, location, and access control information. Rather than having many local configuration files, applications and hardware should store their configuration information in the Active Directory. The Microsoft Management Console (MMC), currently available with Windows NT, will take advantage of this store to make it easier for an administrator to configure remote resources under Windows 2000. Application installation programs will be able to take advantage of the Installation service, which handles many tasks previously handled by the installation programs directly. The advantage of bringing the operating system into the installation process is that it can more easily track the status of shared libraries and configuration information, making upgrades and program removal more reliable. Plug and Play support for peripheral hardware, introduced with Windows 95, will also be included with Windows 2000. This will, among other things, help lower support costs as well as permit remote configuration of disk partitions (provided, of course, that the storage device is Plug and Play compatible). System instrumentation is improved so that compatible devices can offer the operating system more status information than is currently possible.

Active Directory

One centralized aspect of Windows DNA is of great interest to us in our search for lessons in distributed computing. Windows 2000 Server introduces the Active Directory, a directory service native to Windows.

Simply stated, a directory service is a central, hierarchical store of information, intended to be read by network clients. A directory will typically contain information about the configuration and capabilities of networked machines. Generally speaking, a directory contains information that is frequently read, but seldom written. As a result, an effective directory is optimized for read access. Although the Active Directory is a proprietary implementation, it supports the Lightweight Directory Access Protocol (LDAP), an open standard protocol for querying directories. Active Directory supports both COM and C-language APIs for programmatic access.

An active directory instance is global to its domain, although any authorized directory server in the domain may take ownership for authoring purposes. The directory itself is replicated between the primary server and backups on the domain. Directories may forward requests to trusted directory servers in other domains. This helps the directory service scale to enterprise levels by distributing the load across multiple servers. Properly done, the majority of directory queries from clients will be satisfied by directory servers, close — in network segment terms — to the originating client. As we will see in Chapter 2, the Active Directory is a valuable means of describing the application services on our networks.

Transaction Processing

Transaction Processing (TP) monitors are a familiar fixture in enterprise computing environments. Mission critical applications such as banking need the reliability of transactions and high-throughput computing. The Microsoft Transaction Server (MTS), which is bundled with current versions of Windows NT Server and also available for free download as part of the Windows NT 4.0 option pack, is an attempt to provide these services on PC networks using an object-based interface. TP monitors are well-known fixtures in mission-critical mainframe applications and MTS was developed to help programmers write such scalable, mission-critical applications for the Windows platform.

Applications are built as packages of components, and it is MTS that breaks down the tasks performed by these applications into functional blocks, which are implemented by these components. So, to deploy an application on MTS, the administrator registers the components with MTS and defines the security roles for users of the application. MTS is then able to stand between a client application and the COM runtime, and perform scalability enhancing services like resource pooling, as well as distributed transactions. Note the similarity between MTS and the description of interceptors in COM+. MTS is the means of implementing transaction services in COM. When language tools support transaction attributes in COM+, the functionality of MTS will be seamlessly tied to components.

The transaction services provided by MTS can be distributed across multiple databases. MTS uses the Distributed Transaction Coordinator (DTC) service for its transaction services. DTC was originally developed for Microsoft SQL Server to permit transactions across multiple databases. Each database engine implements the local functions of a distributed transaction; MTS (through DTC) merely orchestrates the semantics of coordinating the transaction across multiple databases. Transactions are one of the most useful tools in relational databases. When two or more database commands are enlisted in a transaction, should a single operation fail any commands that have completed are rolled back, the database is restored to its state prior to the transaction. If all operations succeed, however, the state is committed – the database permanently reflects the results of the transaction.

In a distributed system, we will often wish to use data from more than one database; indeed, we may wish to use databases from different vendors. A store, for example, might draw inventory from a regional warehouse. The program controlling this might update an Oracle database at the warehouse and a SQL Server database at the store. Obviously, we do not want to subtract inventory from the warehouse database if the addition to the store database fails.

That is where distributed transactions come in, allowing a programmer to declare a transaction in which operations take place on more than one database. It is the DTC that ensures the same data integrity across databases that conventional transactions offer within a single database.

For more information about MTS check out Professional MTS MSMQ with VB and ASP from Wrox Press, ISBN 1-861001-46-0

Message Queuing

Although most of the applications in this book are implemented with COM, some tasks require an asynchronous message-based service. A message, in this context, is simply an arbitrary data structure that we wish to forward to a remote computer. Like an email message, we do not want to hold program execution while we wait for the remote machine to respond. Instead, a store and forward mechanism allows us to asynchronously pass messages. The sending program is able to continue once it is sure the message has been committed to a queue. This is useful in high volume situations such as e-commerce engines. Requests are sent as messages to the engine, and, once sent, the client is free to continue even when the server has a backlog. The backlog can then be cleared when demand drops.

The Microsoft Message Queue Server (MSMQ) provides for reliable communications through message passing. Copying the structure to a local queue sends a message. If the local machine is unable to connect with the desired recipient machine, the message remains in the queue until it can be sent. Once a message arrives on a remote machine, it is again placed in a queue, where it remains until the receiving application can consume the message. Providing a second form of queuing further enhances reliability. Messages that must survive a system shutdown or crash are automatically written to disk when placed in a queue and can consequently be recovered following a restart. In cases where the programmer wishes to trade reliability for throughput, however, MSMQ can be instructed to maintain the message in memory without writing it to disk. MSMQ even allows for the inclusion of messaging with the context of a transaction, extending the integrity of database transactions to include messaging.

Reliable messaging and messaging transaction present difficult choices for system designers, particularly when used across a high latency network like the Internet. You must balance the need for integrity and reliability against the substantial performance penalties imposed by these features.

MSMQ is hardly the first general-purpose messaging middleware package on the market. In fact, products like MQ Series from IBM and DEC MessageQueue are available for numerous operating systems. Because of the popularity of messaging middleware in the implementation of legacy mission-critical applications, there is a high degree of interest in integrating MSMQ with other messaging systems. Level 8 Systems offers products permitting MSMQ to be bridged with other messaging middleware products.

For more information on interoperability, see the Level 8 Systems home page at
https://www.level8.com

Universal Data Access

Microsoft's goal of unifying access to disparate data stores has evolved through a number of technologies and names. Open Database Connectivity (ODBC), is a platform standard API for addressing relational databases (although ODBC drivers are also available for non-relational stores like text files and Excel spreadsheets). OLE DB, Microsoft's COM-based successor to ODBC, was a significant step forward. It offers a standard API for any data store capable of representing data in a rows-and-columns format, and the API was implemented as a COM object model. Subsequent to the initial introduction of OLE DB, efforts were made to provide efficient access to remote data sources. Many acronyms grew up around OLE DB. Eventually, these confusing terms were brought together under the name Universal Data Access (UDA) in Windows DNA. Clients use ActiveX Data Objects (ADO) as the client interface to Universal Data Access. Beneath ADO, UDA provides a number of abstract services. The simplest UDA components are data providers, representing a single physical store. Service provider components use one or more data providers to offer more complicated features such as cross-source joins. Command processors take some arbitrary command in a data retrieval command language, typically SQL, and use the service and data providers to satisfy the request encoded in the command.

Universal Data Access is of special interest to Web developers because of the way it offers efficient access to remote data through a connectionless protocol (HTTP). Data retrieved from the database server is packaged as a block of MIME-encoded binary data and passed back to a UDA component on the client. That component converts the MIME-encoded data to a local cache. This cache is a disconnected recordset that permits the client to work with the data without maintaining and using a persistant connection across a potentially low speed network link. So, subsequent requests (for example to scroll) from a client ADO component are satisfied through reference to the local cache. Only when the cache must be refreshed is another connection is made with the server.

Windows DNA and Current Practice

What does this mean to us today? Windows DNA is the least futuristic of the three technologies we shall examine. Major pieces of the initiative are either available today or are evolutionary enhancements of currently available technologies. We can — and will — actually work with some of the technologies that fall under the Windows DNA umbrella.

Perhaps the most important lesson of Windows DNA is the importance of distributed programming to future systems. Although current practices like client-server computing and Distributed COM are properly considered forms of distributed computing, bringing the integrity and reliability achieved in monolithic applications currently requires substantial custom programming. Windows DNA not only delivers standard operating system services to address these concerns, but it makes them easily available to application programmers. This encourages system designers to build applications that fully utilize the distributed resources of a modern network. The control of the old centralized computing model is blended with the power and personalized access to resources in the PC model to achieve something more powerful than either model alone.

Windows DNA is component based at every level. Programmers who embrace the model of building software by coordinating discrete software components can easily gain the benefits of Windows DNA. The introduction of a directory service highlights the importance of a central store to which applications can go to discover and locate resources at runtime. It is not hard to see that if applications can dynamically discover resources and enlist them to accomplish their tasks, distributed applications will become more fluid. They will locate, access, and release the resources available to them on the network as and when needed.

Millennium

Unlike Windows DNA, Millennium is a research project being conducted by Microsoft's research division. This project is an investigation into the issues surrounding an inherently distributed operating system. It assumes inherently distributed applications — the resources of the network not only enhance the performance of an application, they are required for its execution. Where Windows DNA is a vision of improving and extending existing, PC-centric networks, Millennium is a vision of a radically different computing environment. A system organized around Millennium's principles makes dynamic and transparent use of computing resources without prior configuration by the system designer. Windows DNA requires programmers and system administrators to manually partition systems for optimal performance. Such partitioning is explicit and obvious to those who use the system. In Millennium's vision, the ability to distribute computing tasks is implicit in the operating system. Individual computing resources are important mainly in the context of the network.

Like all long-term research projects, Millennium is subject to evolution and change. For the latest information on the project from Microsoft Research, see the Millennium Web site at https://www.research.microsoft.com/sn/Millennium .

Millennium's Goals

The nature of a distributed operating system in Microsoft's long-term vision can best be understood through the goals of the Millennium project:

Automatic distribution — the operating system should determine how tasks in the application are distributed and make remote resources available as local resources
Scalability — the operating system should consist of one logical system image, and any application should be able to scale as needed within the network
Fault tolerance — the system and applications that run on it should adapt dynamically to failures and loss of connectivity without loss of data or functionality
Self-tuning — the system should reallocate resources and redistribute application tasks based on observed application performance
Self-configuring — the system should automatically incorporate new resources as they are connected to the network
Security — despite a single system image, users and service providers can specify access control permissions for resources and users; the system should also allow non-hierarchical trust domains
Resource control — users and service providers can place constraints on resources belonging to different trust domains for reasons of security

While some of these goals can be met with Windows DNA services and clustering, self-tuning and self-configuring reach their full potential under Millennium.

These goals imply certain approaches. Four key principles may be derived from the Millennium project.

Abstract and automatic distribution of tasks — the system performs distribution tasks, freeing programmers to focus on the problem at hand. Programmers and applications are unaware of the location of resources enlisted by the system on their behalf.
Dynamic enlistment of resources — specific hardware resources and software services are bound to the application as needed at runtime.
Abstract locality — neither applications running on nor programmers using the system know where computations and storage reside; the system acts as a middleman, locating required resources.
Automatic performance tuning — the system must continually monitor the execution of applications and dynamically adjust the distribution of tasks in order to optimize system and application performance.

Web programmers may look on Millennium as an idealized version of the Internet. Presently, we manually search for servers and sites offering the data and services we require. The Domain Name System abstracts locality, converting a named resource into a physical location. Although the Internet does not self-tune its performance, the distributed nature of routing provides a crude sort of dynamic adaptation.

Millennium Prototype Projects

Millennium is investigating the issues of distributed operating systems through three prototype projects using some current technologies. Each of the three prototypes uses a different technology for decomposing applications into tasks:

Coign which uses COM components
Borg which uses Java classes
Continuum which uses COM+ components and is more dynamic than Coign in its approach to partitioning

As this book goes to press, Borg is no longer listed on the Millenium Web site. Whether this means Borg has been abandoned, subsumed into one of the other projects, or simply removed due to the on-going law suits between Sun and Microsoft is unknown. It is listed here because it reflects work previously done by Microsoft and illustrates some important points.

Having seen the aims of Millennium, let's take a look at how each of these projects uses a different approach to meet the goals of the Millennium Project, and then see what we can learn from the ways in which these projects are trying to advance distributed networking.

Coign

The Coign prototype implementation performs automatic distribution using COM components as the fundamental means of partitioning an application. Any application built from COM components, whether designed as a distributed application or not, can be partitioned by Coign. An important point to note is that the Coign system does not need access to the source code for the components, nor do the components have to be built with special libraries linked to them. Coign watches while the user works with the application, notes how often components are accessed and how they interrelate, then develops a scheme for distributing the components across available computers on a network.

A Coign developer runs a tool to insert run-time instrumentation into the binary code of the components. Next, he executes the application, presumably running through scenarios reflecting typical application use. Coign builds a model capturing inter-component dependencies. When deployed, a small, residual piece of the Coign runtime code embedded in the component binaries distributes the components in such a way as to minimize component communication costs. These costs reflect network latency and available bandwidth between partitioned components.

At the time of writing Coign did not adaptively self-tune, although users could alter the distribution of components by repeating the modeling process. Since the process does not require the source code for the component, the profiling process does not compromise the intellectual property of the application.

Coign runs on Windows NT and was tested with three applications, a commercially available photo editing application, a word processor, and a component-based intranet application intended as an instructional resource for developers. The developers of the last application manually partitioned the application. Re-partitioning using Coign reduced inter-component communication costs by 54%.

Borg

This prototype aims to provide a single system image (SSI) of a distributed Java virtual machine (VM). Java classes are distributed beneath the SSI VM to enlist network resources in the execution of a single Java application. Programmers of a Java application running under Borg wouldn't derive their classes from any special base classes, make special distinctions regarding distribution, or otherwise know in any way that the classes their application would execute in a distributed environment. A single system image VM provides the benefits of distributed computing without explicit programming effort. A prototype of Borg has been successfully demonstrated.

One reason why Borg may no longer be a separate project is that the Microsoft JVM can expose Java classes as COM components. Once that is done, there is little difference between Coign and Borg.

Continuum

Like Borg, Continuum is a single system image prototype. Whereas Coign produces a static distribution of computing resources, Continuum uses the features of COM+ to allow it to dynamically migrate components from one computer to another while the application is running. Continuum would allow an application to be continually re-tuned in response to changes in the network environment.

Observations

These three prototypes share three fundamental principles:

Automated partitioning — Each prototype examines how an application uses components to accomplish its goals, then compares this to the computing resources available to it and partitions the application into groups of components. The groups are distributed to computers that can efficiently host them.
Self-tuning — The prototypes decide the optimum allocation of available resources without operator intervention. Continuum goes a step further, performing this task while an application runs. If successful, Continuum would continually balance network resources as devices are added and removed, or if they are heavily loaded.
Abstract locality — A user of these systems sees a single logical application and computer. Only the system knows where the resources that make up the application actually reside.

All three projects move application development to a higher level of abstraction, pushing physical network considerations into the runtime system. All are SSI implementations in that the user experiences a single, monolithic computing entity. Coign attempts aggressive optimization of commercially available applications by splitting up locally hosted COM components and efficiently distributing them. Continuum goes a step further, although it requires that the application be built from COM+ components and therefore cannot be used with today's commercial applications. Continuum performs the same task of automated partitioning as Coign, but where Coign does it once in a test setting, Continuum performs it dynamically in a production setting. As available resources change, Continuum moves components to balance the system.

Millennium and Current Practice

Today, components and abstract locality are the most relevant features of the Millennium project. The partitioning and self-tuning features are exciting, but well beyond anything we will be able to do in current practice. Millennium, at its heart, relies on software built substantially from components. This is something we can do today, and which is becoming increasingly popular. In fact, Coign was tested in part on commercially shipping software. If Millennium's view of the future is correct, we can start down this road immediately.

Abstract locality is a bit more difficult to approach. Millennium carries this to an extreme; not only does the user not know where resources are located, he can only presume that the system is distributed, because Millennium presents a single system image, hiding the location of its resources. This is beyond current production practice. One lesson we can learn, however, is the value in delaying the firm commitment of resources until they are needed. Monolithic applications include resources within them, and client-server applications often explicitly specify the location of resources in their configuration, while Millennium hides the location of resources. Using directory services, you can specify the nature and location of resources in such a way that well-written applications can determine what they need and where it is located on a just-in-time basis.

Jini

Jini is Sun Microsystems' experimental foray into providing highly dynamic network services to heterogeneous computing devices. Jini begins with the Java programming language but goes far beyond Web page applets. Jini is an architecture for distributed computing, influenced by the Linda system pioneered at Yale University (https://www.cs.yale.edu/HTML/YALE/CS/Linda/linda.html).

Linda is a programming language with supporting tools that allow programmers of distributed applications to specify how to combine remote resources in such a way as to accomplish the goals of the program. Jini aims to provide a means for users — human and computational — to discover services provided by resources on a network. 'Resources' not only embraces PCs, but also standalone storage devices, printers, and other network-enabled appliances. Jini federates these groups of devices into systems that can be viewed, managed, and accessed as easily as the local resources of a PC. By this Jini means that combinations of resources are enlisted at runtime to accomplish specific tasks. The notion that they can only accomplish the task by combining resources is what federation means. The core resource element in a Jini system is a service. Do not confuse this with the definition of service provided in the introduction. Jini services are very specific, and largely pertain to narrowly defined network or operating system resources. Our meaning is closer to application modules. Clients — users and devices — search for Jini services by contacting a well-known lookup service, which in turn can reference other lookup services or indicate the source of the service desired by the client. Clients then lease a particular service for a specified duration. Clients accomplish tasks by leasing all the services required to accomplish some task.

Jini assumes that all devices support a Java Virtual Machine (VM) and that all services are provided by Java components. This enables Jini to build on the services being rolled out for Java, and allows components to travel between Java VMs. Unfortunately, this means that all prospective Jini systems must be built from programs written in Java, thereby eliminating direct use of legacy systems written in other languages.

System Services

Like the designers of Windows DNA, Jini's architects recognize the need for system services that are more advanced than basic operating system services, like file and print services. Jini begins with the services offered by common Java technology:

Java VM — execution and runtime services for downloaded byte codes
Remote Method Invocation (RMI) — invoking the methods of Java classes on remote machines
Security — control over what actions downloaded code can take
JavaBeans — a component software technology
Java Naming and Directory Interface (JNDI) — network directory services
Java Transaction Service and API — specifications for building transactional services in Java

I'm using a very broad definition of 'services' here. Sun sub-classifies these services into infrastructure, programming model, and services. The distinction is not important for our purposes. Suffice to say we are only concerned with the features offered by Jini/Java to the builders of applications and business systems.

Jini adds enhancements to the Java platform as well as new services:

Extended RMI — JDK 1.2 enhancements to RMI to fully support the Jini vision, distributed garbage collection for example
Discovery — a protocol for advertising or finding the availability of a service on a Jini system
Distributed Security — access control lists for users of a system and the rights granted to them
Two Phase Commit — reliable database commits between distinct databases
Events — a distributed mechanism for signaling events on remote objects
JavaSpace — the server that enables clients to locate Jini system services and reserve access to them

As you can see, Jini is more than simply communicating between components on remote machines. It builds applications by forming federations of network services, enlisting them in the common cause of some application task. It provides a robust framework of platform capabilities that programmers can draw on in a just-in-time basis. The full system is extremely dynamic and capable of adjusting to fluctuating network configurations. Jini, therefore, marries the advantages of component software to the advantages of reliable access to network services to raise application programming to a higher level of productivity and performance.

JavaSpaces

The provision of services in Jini is accomplished through the flow of objects between servers and clients. The entire Jini model relies on the location and negotiation of services on behalf of a client. A client application must find a service it requires, reserve that service, and use the service implementation provided by a server. JavaSpaces servers act as the broker matching clients with services. They are the middlemen between clients and service providers, providing a common clearing house for the entire network.

A JavaSpaces server contains entries, each of which is a Java object implementing a well-known interface. An entry contains fields, each of which is also a Java object. Servers offering a service in a Jini system write an entry to a JavaSpaces server, thereby copying the objects into the server. These objects are proxies for objects maintained by the service provider. Consider the following, highly simplified Jini system:

Locating a Service

A client desiring service may locate a provider in one of two ways. Firstly, a client may request that a JavaSpaces server notify it when a particular service becomes available. This notification is accomplished using the distributed event service. Alternatively, a client may actively search for a service by presenting a template to a JavaSpaces server. A template contains entries and wildcards. If the entries in the template match entries residing in the JavaSpaces server, a match is found. An entry in the server and a template entry match if the two entries are of the same Java class and if the public fields of the entries contain the same values. Wildcards are strictly a 'don't care' entry placeholder.

Templates are presented using one of two operations: read and take. In a read operation, matching entries are copied to the invoking client without changing the entry on the server. In a take operation, the entry is removed from the JavaSpaces server after it has been copied to the client. Continuing with our simple system:

Accessing a Service

Once the client has obtained an entry from the JavaSpaces server, it has access to the service it needs. The entry obtained may implement the service locally, or, more commonly, act as a proxy, communicating via RMI with the server that originally wrote the entry to the JavaSpaces server. Finally, our simple client obtains services:

These are just the simplest elements of JavaSpaces. Large Jini systems require a number of features to support scaling to very large environments and provide for a robust, nonhierarchical system. Jini also provides a number of advanced features that are beyond the scope of our discussion. Two examples related to system scaling are JavaSpaces server replication and service lookup forwarding. JavaSpaces servers may replicate their entries to other servers in the interest of keeping lookups localized within network segments, which also improves performance and reliability when network segments fail. A JavaSpaces server may also forward a lookup request to another JavaSpaces server to facilitate location, while objects may break a problem down into smaller pieces and do lookups of their own using additional JavaSpaces servers. Overall the JavaSpaces specification provides for a powerful system based on the brokered flow of objects between hosts in a Jini system.

Leases

A fundamental problem with distributed systems is that one party to a remote computation can fail, or the network connectivity between the two parties can be broken. When a client is lost, the server may maintain unused resources indefinitely. A lost server may cause the client to wait indefinitely for service. A mechanism must be added so that parties to a distributed computation can detect an unannounced loss of service. Some distributed systems use periodic polling to determine if the parties are still in communication. Others set a 'time to live' in the granted object. Jini uses the concept of leases.

A client guarantees access to a service by obtaining a lease. A lease is granted for a particular duration, typically negotiated between the client's desires and the server's willingness to maintain the resource. While the server will typically grant the lease for the term desired, a client may make an unreasonable request, or a server may be more constrained than expected, so negotiation allows servers the opportunity to downgrade the duration of what is asked. Negotiation is simpler than it sounds. A client asks for a particular lease, and the server agrees or offers something less. Using duration as the unit of measure for a lease rather than an absolute time avoids the need to globally synchronize clocks in a Jini system; indeed, it makes the federation of Jini systems possible. Consider a Jini system in which all clocks have been synchronized. Now, suppose a client wishes to have a service in another Jini system, if the two systems have differing clocks, who synchronizes with whom? Using duration eliminates the need to make sweeping synchronization changes. Once obtained, a client may renew a lease if the service is still needed. Leases may be exclusive or shared depending on the nature and implementation of the service. Clients may also cancel leases, permitting the early cleanup of resources.

An additional feature of leases is that they allow brokers like JavaSpaces, or application-specific entities, to negotiate leases on pools of resources in the interests of efficiency. A server can pre-allocate a pool of objects in order to respond faster to new service requests. When a third party claims a resource holding the lease, the lease simply travels with the resource, and the broker need not further concern itself with the resource's leasing arrangements.

Jini and Current Practice

Jini seems to have struck a nerve in the networking community. In addition to the interest in all things Java, Jini has attracted particular interest in its ability to connect dissimilar devices on a common network. Even if the more advanced features of Jini — the ability to federate distributed resources for the purposes of an application — are never realized, this feature is likely to be pushed forward into the marketplace by Sun.

What we as application developers can take away from Jini's example is the idea of advertising services. You can see how this would complement the idea of finding the location of a resource as needed. If the provider of some sort of programmatic logic were to advertise it in a form that other applications could understand, applications could ask for the location not of a named program or device, but for the location of a program or device that accomplishes a particular logical task. Jini focuses on specific devices and fine-grained components. We can do something a bit easier by finding a way to classify our server-based programs according to the logic they carry out.

Back to Reality

Hopefully, this brief excursion into the future was exciting. Unfortunately, upon returning to mundane reality, we still have to implement Web-based systems and we only have current technologies, languages, and networks.

Some of the technologies we've seen, such as component software, are currently available, so we will make extensive use of them. Others, like automatic component profiling and distribution, must wait for future operating systems and platforms. Somewhere in between lie functions we would like to have for developing Web based applications and which can be implemented using currently available programming techniques.

So that we can properly identify where we want to make improvements, we should take a quick look at some of the weaknesses in our available technologies.

Where Are Our Present Weaknesses?

Before embarking on the task of identifying the functions of these future technologies that we can implement, we should ask why we need additional services at all. Many powerful Web and intranet sites exist using nothing more than the basic, standard internetworking standards — HTTP, CGI, HTML, and so forth. So why do we need new techniques?

Dealing With Increased Complexity

Powerful sites addressing serious problems are easily as complicated as a stand-alone application. Moreover, they have the added complexities inherent in distributed systems. Unmanaged complexity is a weakness of distributed systems. Resources become less and less useful as knowledge of their meaning and capabilities is lost to later-deployed clients.

Because Web-based applications are easily fielded and require no one's permission to add to the Internet (other than the obvious requirement of obtaining a domain name), servers and services quickly blossom and mutate. Presumably someone understands what they can do, but there are many more clients who could use them but who will never know these particular capabilities are available. Functions will be needlessly reimplemented again and again simply because no mechanism has been defined to advertise what is available. This is barely tolerable in the case of the public Internet; but it is a complete recipe for waste and disaster in a corporate intranet.

Inflexible Applications

Applications are often written with a particular client in mind. Alter the needs of the client and you either have to re-write the server or create a new, highly similar service.

Duplication of Data

New servers often duplicate large bodies of existing code because of minor changes in needs and specifications. This duplication of code can be seen as wasteful.

Moves Towards Automated Web Tasks

If we wish to automate a previously manual Web-based application, we have a new problem. Our previous server generated HTML for human consumption, but when the consumer is an automated application, HTML is far harder to consume.

Distributed Development and Implementation Teams

Disasters frequently occur when services are provided by some organization not under the control of the prospective client.

For example, you may be building an extranet supply chain application, in which case you must rely on an external partner for some of your application's services. When coordination is informal or nonexistent, services will move, change, and disappear. The administrative overhead of keeping pace with the changes rapidly snowballs. We need to add techniques and services that let us build flexible, robust, fault tolerant distributed systems that can change over time along with our needs. More importantly, we need to do this without requiring human coordination.

Hopefully, the future systems we have just examined will give us some ideas we can use to solve these problems.

So, What Can We Learn From The Future?

It was all very interesting looking at the aims of these research projects, but we cannot wait for them to come to fruition, especially not when we live and work according to the accelerated rate of Internet time. You aren't likely to have bought this book for a prediction of the future technologies, so let's see what the common threads of these different efforts are so that we can crystallize our thoughts on how we are going to build our distributed cooperative applications today.

The three systems we have considered — and they are by no means the only visions growing in the field of computing — are highly dissimilar:

The Microsoft systems use COM as their bedrock, while Jini is all about Java.
Millennium is organized around single system images, while Jini looks for highly fluid systems of dissimilar resources.
Windows DNA is in the process of fruition, using many tested technologies, while Millennium and Jini are clearly in the realm of blue-sky research.

Common Threads

Having noted some differences, these systems face the problems common to all distributed systems and draw their solutions from the same existing research literature. Not surprisingly, then, some common threads are discernable. The main five are:

Components as the building blocks of applications
Abstraction of the location of services, and the dynamic discovery of location at runtime
Dynamic use of resources for a task, and the ad hoc release of resources
Formal description of desired resources at runtime
Distribution of the programming task across multiple resources of the network

Let's take a closer look at each of these threads in turn, what they mean and how they affect us developing distributed cooperative applications. Unsurprisingly, these common threads address the weaknesses we have just outlined. So, let's look at the problems that each of these threads is trying to solve, and how they aim to do so using these common threads.

Component Software

Problem:

Object oriented computing swept over commercial practice in the 1980s with a mixed record of achieving its claimed objectives. Its model, however, proved useful and powerful. Data could now be packaged with the programming logic needed to properly handle it in a single, well-defined entity. Complicated programs could be built from the interaction of simpler packages operating black-box style through compiler-enforced interfaces. In practice, however, this was seldom realized. Access to source code tempted many programmers to modify code rather than use inheritance as rigorously as they might otherwise have done. Even when code was reused through its interfaces, language dependencies meant implementations in one language, e.g. C++, could not be used by applications written in another language, e.g. Visual Basic. Vendor specific dependencies further restricted the scope of reuse.

Solution:

In the slightly mutated form of components, object orientation is proving to be a big boon to programmers. Language neutral component technologies such as COM permit the broadest possible reuse. A genuinely useful third-party component quickly finds use in a variety of applications. All the systems we have considered are built from components. Applications are built by assembling and coordinating the activities of components. These components are built, compiled, and tested separately from the using application, often by third parties. The result looks to the user like a single application, but is really the assembly and reuse of standard components. Complexity is managed by allowing the programmer to focus on the task at hand, relying on the components for large bodies of detailed code implementing lower-level operations. Clients and servers are coordinated through agreement on programmatic interfaces.

Abstract Location

Problem:

When you first learned to program a client–server application, you likely hard-coded the network location of the server into your code. The name of the server application and its specific network address was written into the client. It became apparent that this would not work in a real application, since servers can be renamed and addresses often change. In response, you might have adopted some sort of initialization scheme. You may have asked the client to read the name and address from a text file, or asked it to look up the same information from the Windows registry. Perhaps this information took the form of command line parameters needed to start the client executable. Clients and servers are less strongly coupled, but this will not suffice for the highly dynamic systems we envision. Using this approach, the information is provided in arbitrary ways. The client programmer must know where to look for the information. There is no standard place to look for the information or — in the case of the registry — standard way of naming these places within a well-known data store. Moreover, knowledge of what a particular server offers is assumed. Taking this approach, clients were therefore built largely in parallel with their servers, or, at the very least, built by a team with intimate knowledge of the server.

Solution:

Microsoft wants us to use Windows DNA as the infrastructure for enterprise-scale, mission-critical systems. Consequently, we cannot count on initializing our knowledge of resource location by restarting our clients. The Active Directory in Windows 2000, or some other vendor's LDAP-capable directory service implementation, provides a persistent yet readily queried store for configuration information, including the location and parameters of our servers, which saves us from having to hard code our resources. Information entered by network administrators is copied at intervals to other directory servers, ensuring every client can get the information from a local source well-known to it. Clients can then turn to this source for information about the network and its resources.

Millennium takes this a step further. It doesn't just allow you to move services around the network and reference them from a directory service, rather it is built on the idea of hiding the location and nature of these services from you! It goes beyond the concept of directories, in that it does the querying and location for the using application. The result is a single system image rather than a map to distributed resources. Although at the time of writing it was not yet possible with current production technology, Coign may someday allow for the dynamic partitioning of resources, so the system demands the ability to locate services on an ad hoc basis.

Meanwhile, JavaSpaces is an indispensable piece of Jini, and its reason for existence is the dynamic discovery of and connection to services by clients at runtime.

Dynamic Resource Enlistment

Problem:

A fundamental assumption of all these systems is a reliable network that is always 'up' built from individual nodes that come and go with little or no coordination.

Stand-alone applications linked third party libraries of compiled code together in an iron-clad contract for life. The introduction of dynamically loaded libraries added some dynamic acquisition to the programmer's arsenal, but often applications were compiled with early-bound knowledge of services and loaded libraries when the application started. Going to a Web-based programming model forces us to be more fluid in our thinking, open to unsettled environments. When we request a service from an HTTP server it may not be available. Web servers that permit us to maintain a measure of state in an inherently stateless protocol require us to periodically renew our interest in the server. The contract between a client and a Web-based service is not so tight as in older models, and requires more flexibility on the part of all parties.

The HTTP protocol is stateless by design. A client cannot therefore count on a server-based resource being available at some time in the future simply because it was available for a prior request. Resource location, consequently, must be performed as close to the time of use as possible, and client-side code must be prepared to take appropriate action if a desired service is unavailable.

Web applications that span multiple server-side pages or scripts require that servers offer some kind of session state information for each client. In a small-scale ASP application, this is held in the Session object. The server, like the clients, cannot know whether the client is still interested in the session (clients leave abruptly and fail unexpectedly, after all), so all session state mechanisms are time dependent. If a client relies on server-side state, it must periodically renew contact with the server to ensure that session state information is not destroyed. These are factors you must consider when designing distributed applications using Web technology.

Solution:

Component software and transaction services in Windows DNA provide for the difficulties of enlisting multiple resources. Should a particular service implementation be offline, a client application may try to find other implementations through the directory service. Should all implementations of a needed service be down, then an application can abort the operation (or rollback a transaction). The Web-based interface implies script-based clients, which call for services dynamically and employ late binding of component interfaces.

Millennium's single system image metaphor implies dynamically enlisted components, else it wouldn't be fault tolerant. After all, with SSI, the client is denied knowledge of network resources (as the Millennium runtime is solely responsible for coordinating distributed resources), so the Millennium system must be able to dynamically replace a lost resource with a newly created replacement.

Jini goes a step further. Its programming model, you'll recall, involves solving problems through the federation of services and the flow of objects from host to host. Efficient use of resources implies that clients acquire leases only as needed and for the minimum duration necessary. Applications on Jini systems must inherently be programmed to solve problems through the dynamic creation, flow, and destruction of resources.

Description of Required Resources

Problem:

Given the need to acquire services as we need them, how do we tell the server what it is that we need? In common Web-based applications, the service is implied by the URL requested. An ASP or CGI script is known to implement a particular service; by requesting the resource, you implicitly request the service. Nothing, however, prevents a new service from being created using the old URL, and the result is a broken application. URLs are intended to name a resource; any information regarding what is expected of the client and server — the contract between them for service — is strictly implied. A URL does not explicitly define an interface to a service. Something better, and more descriptive, is needed.

Solution:

Both Windows DNA and Millennium have COM at their hearts, so resource description is based on interface. When you want a particular set of functions, you ask for the interface that implements those features. If you have an interface pointer, you expect a particular set of features and behavior. This is the programming equivalent of a legal contract in the real world. Any server implementing the interface is bound to the documented semantics of the interface. If a designer wishes to modify the interface or its semantics, he is required by the rules of COM to offer a new interface. Millennium's Borg prototype and Jini use Java, which also works with interfaces that are implemented by Java classes.

Task Distribution Across Networked Resources

Problem:

The whole point of distributed applications is to spread the programming load across the network, placing responsibility for accomplishing tasks with the host best able to serve the need.

Solution:

Windows DNA takes some steps in this direction, although most task distribution must be manual. MTS and clustering services in Windows DNA allow the creation of multiple tier applications, leaving system architects to make appropriate decisions regarding task allocation.

Millennium's Coign takes the opposite extreme — this prototype has task distribution at its core. Moreover, this partitioning is largely autonomous.

Jini falls somewhere in between. Like Windows DNA, task distribution is largely a matter of system architects making decisions at design time about the hosting of services. With the ability of Java objects to flow between machines through JavaSpaces, however, programmers are given better tools with which to alter partitioning. Services may be manually moved to another host, with the new host broadcasting its availability through the JavaSpaces in its Jini system.

So, we have seen these five common themes through the different systems we have studied. How can these ideals be seen in terms of what we can actually implement today?

What is Practical Today

Clearly, the practicality of implementing these common threads with today's technology varies depending on what we are talking about. Some are proven capabilities, some are possible in rudimentary form using a combination of existing technology and programming conventions, and some things will simply have to remain in the realms of research experiments for some years to come. Let's consider each of these problems and solutions, which we have met in the form of the above common threads, and see what we can do today with today with commonly available operating systems and languages.

While we have been stressing the fact that these ideas are taken from research projects, you will no doubt have seen some parallels with technologies that are available today and techniques that we can use to achieve similar ends. This is where we actually get to see how these projects can influence our development directly today.

Where and How We Can Use Component Software

The three future technologies we examined have components at their core for a reason: components offer many benefits. During the design phase of a project, we obtain the following benefits:

Abstraction — high level goals are accomplished by coordinating the activities of a series of 'black box' components. The implementation is deferred for later, or even passed off to a third-party for development or purchase.
Delegation — at the highest level, you can specify an interface in terms of preconditions, semantics, and results. The task of implementing the interface can be delayed until later.

During implementation, we see more benefits from components:

Cost — encapsulation reduces the number of bugs due to closely coupled 'spaghetti' code. Errors in data integrity can be avoided because the data is accessed solely through code methods intended to enforce the business rules that define the nature and integrity of the data.
Availability — A thriving market of third-party, general purpose components exists. We buy implementations of common tasks like charting and grid displays rather than build them.
Reuse — once a component implementation of custom business logic is built and tested, other applications dealing with similar aspects of the business simply use that component. Not only do they avoid redoing work accomplished by others, but all users are assured of a consistent implementation of the rules of the business.

Component software is all around us. Microsoft Internet Explorer is an application built completely from software components and many Windows applications use components to some extent. In addition to COM, JavaBeans is another viable component software technology that is already used, and if we extend the definition to primarily distributed technologies, we can include CORBA in our list. There are also some other, lesser-used technologies such as IBM's SOM and the late, lamented OpenDoc technology stemming from the Macintosh world. Component software aids rapid application development though the ability to reuse proven implementations without resorting to the source code. So, the issue is not whether we can use component software in our designs, rather where and how.

Interoperability is a key issue. COM is primarily a Microsoft Windows-only technology, and despite the existence of a number of commercially available ports of COM to other operating system, the vast majority of COM-based projects are written for Windows. CORBA, meanwhile, suffers from slow growth, perhaps due to the fragmented implementations of the common CORBA standard. Finally, Java is just getting started, and cross-platform portability is a question best answered with a tentative "Yes, but…" reply.

The guideline for practicing Web developers, then, must be to restrict the use of components to a tier wholly under the control of the development team to remove problems of interoperability.

COM components can be a tremendously valuable asset to server-side development, or in cases where the client is known to run on a platform where COM is available. We will make extensive use of pre-built COM components in this book in just this way.

Avoid using a given component technology across boundaries, and do not assume that clients and servers share a common component technology. Use components within the 'black boxes' of your networked system to speed development, and leave distributed component technology on public networks for the future.

Distributed component technologies like DCOM and CORBA are best used within a controlled environment like a low latency LAN. In wide area networks, latency and heterogeneous technologies rear their incompatible heads. In such cases, you need a more flexible, platform neutral mechanism for allowing resources to interact.

Isolating Applications from Configuration Issues

The future technologies we looked at allowed for the dynamic discovery of resource location through reference to a single well-known point of reference. The Active Directory in Windows DNA, the SSI in Millennium, and JavaSpaces in Jini implemented this task for us. They provided a flexible and open ended repository for application-defined information. All clients on the respective systems have access to the repository through a common API. In current practice, the best solution to this problem is directory services as we saw in DNA.

Directory technology is just coming into common use. LDAP — Lightweight Directory Access Protocol — is a valuable open standard for access to directories in a Web environment, but it is still new. Directories, as we will see, are so much better than initialization files and other schemes in the construction of large, robust production systems that we will use them in this book when we need to find a given resource. Web programmers should become accustomed to writing applications so that they are location-independent and rely on directory queries to find resources. If you do this, your applications will automatically respond to changes in the network. The network administrator updates an entry, and your application will respond by following the new information to the resource it desires.

What can we put in our directories? Certainly we can discover the network location of services by searching through well-known directories. By joining directories in a hierarchy, we can control the scope of our search. Depending on the importance of a computation, we might want to limit our search for resources to a local workgroup or expand it to encompass all directory servers in an organization. As we expand the scope of our search, we implicitly expand the scope of the resources we might consume. Alternatively, we might know that a service is hosted somewhere within a particular organization foreign to us. In that case, we will wish to go directly to the directory server for that organization with our query.

We not only want to know where something is located, but also just what service it provides for us. The level of the directory hierarchy closest to the resource is best situated to know the current location and condition of the resource. We can provide some limited information regarding the capabilities of the implementing host for the service. There is no established scheme for doing this, however, so we will have to establish some design principles of our own and rely on adopting them throughout the network. Chapter 2 will show you a particular scheme that works with the other techniques presented in this book. Hopefully, our tentative efforts will prove sufficiently useful that standards will come in time.

What Does Dynamic Resource Enlistment Mean Today?

Web developers utilize dynamic enlistment of resources every time they build a client that uses CGI scripts or Active Server Pages. The server has no prior knowledge that a client request is coming and typically does not maintain significant state for the client following the request. Think of a URL as naming a computing resource instead of simply a Web page. The client claims resources based on a name — the URL — then uses them, and loses the resource immediately. HTTP is an inherently stateless protocol, however, and what state is retained in applications depends on a variety of techniques.

Web servers that allow developers to maintain session state information typically do so through the use of client-side cookies and proprietary techniques on the server. But developers cannot assume that all users of their applications will have cookie support enabled in their browsers. Worse, these techniques fail utterly when we try to scale the application through the use of server farms (multiple HTTP servers to which requests are routed round-robin fashion, or according to some other algorithm). Commercial grade servers such as Microsoft Site Server or Netscape Enterprise Server use other techniques to provide a central repository that all the servers can access. Site Server uses a proprietary database with extensive in-memory caching, while Enterprise Server resorts to an LDAP directory server. You can see that we are once again headed in the direction of the futuristic technologies and there is a very good reason for this. Everyone is working from the same body of research literature, and talented programmers tend to converge on the same good answers, differing only in particular details or over differing requirements. So, where does this leave us?

Jini-style leases are plainly outside the scope of today's implementations. This limits the reliability of Web applications that share a significant amount of the computational burden between the client and server. Jini lets us lock in the availability of a resource for a specified period of time. Lacking that, we must employ measures that promise the same reliability.

We will restrict ourselves to conventional, stateless client-server interactions for the most part. Dynamic enlistment, in the sense of late binding of components in scripts implementing services, will also be used. Wherever possible, we would like to take advantage of abstract location in finding our components as well, but the current state of the art limits this. In our case, COM components, the Windows system registry lets us avoid hard-coding the specific location of a component or the name of its implementing DLL, but this is not nearly as good as a central directory service.

How Can We Describe Required Resources?

Defining the specific features and data formats we require of an interface (resource description) is a particularly difficult problem. Component technology relies on the assumption that all parties understand an interface if they share an identifier for the interface. We've decided not to share components between partners on a distributed network so as to permit us to use dissimilar platforms without worrying about bridging component technologies. As a result, we need something better, some neutral means of describing what it is we require of a service. Ideally, we should be able to discover the nature of an interface — learning its contents and capabilities as we go. The artificial intelligence community has made some efforts along those lines, but they are far from standardized. The expert systems community, for example, is working on a knowledge description format (KDF), which will be a standard way to define facts and queries. Fortunately, we don't need real intelligence, just a convention for describing data and referring to data formats in terms of the syntax used to describe them. For example, we might wish to say, "Here is a person's name — last name, first name, middle initial — now look him up and give me his address and phone number. Be sure to tell me what strings make up the street, city, state, and so forth in the address".

In the coming chapters I will introduce you to the Extensible Markup Language (XML), a tagged, text-based scheme for describing data. I will go into more detail in Chapter 3, but for now let's say that XML is entirely text based. It allows users to create new tags very much in the way HTML defines tags. It is a simple and flexible method of marking up proprietary information in a way that allows receiving applications to precisely locate the data they require. As a result, it is rapidly growing in popularity. I will also introduce you to a proposed draft standard for describing XML schemas. These are XML documents that define the permissible tags and sequences of tags in other XML documents that conform to the schema. More importantly, I will present some techniques and conventions proprietary to this book. These will deal with some commonly encountered problems and provide solutions intended to increase the fault tolerance of your Web-based systems. These techniques will be kept simple, as I believe that is the key to their adoption. Agreement on these basic principles by you and the developers of the services with which your applications interact will permit you to build robust applications that degrade gracefully.

How Far Can We Go in Partitioning Systems?

Sadly, the answer here is simple — we are limited to manual partitioning using multiple tier architecture, segmented by the human designers of the systems. This is because the autonomous partitioning promised by Millennium is well outside the scope of current practice, while migrating components such as Jini's rely on a common component software technology. We will gain the benefits of partitioning mainly through the goal of making it easy for servers to cooperate. The techniques described in this book are intended to help you build servers that can be reused by other applications and development teams.

In the sense that you are more likely to partition a system if you can leverage the power of existing servers, we promote partitioning. Where we are able to describe the capabilities of the services hosted by a particular server, we increase the number of clients who can make use of the partitioning scheme. We are not restricted to a single client with implicit knowledge of the scheme; we can open the distributed resources to any client that can read our descriptions. How well these services utilize network resources, however, will depend on the skill and insight of their designers.

We have looked into the future at projects in development now. We have seen the principles that they want to employ in distributed applications. Finally, it is time to articulate a philosophy for building cooperative network applications. To do this we will introduce The 5 Principles for Building Cooperative Network Applications.

The 5 Principles of Building Cooperative Network Applications

Having looked at the ongoing projects for the future of distributed systems, (finding common threads between them and noting the problems they address) we can distill what we have learnt easily into 5 overall principles. The 5 Principles Of Building Cooperative Network Applications will encompass the goals we need to achieve in developing applications that will promote the reuse of application code and the viability of applications in the face of change in needs and applications.

So, here they are, our 5 Principles of Building Cooperative Network Applications:

Applications Will Be Built From Coarse-Grained Services — applications will be implemented by coordinating the discrete results obtained from server-based modules that are larger than individual components and which answer a specific question, problem, or task.
Services Will Be Discovered By Querying Directories — applications will find the location and name of the services they need at runtime by querying a directory. They will ask not for a specific implementation, but rather for any service implementation that addresses a particular question, problem, or task.
Services Will Be Provided As Self-Describing Data — applications will deal with services by exchanging structured data. The data will be written according to vocabularies defined for the problem or task that the service addresses.
Services Will Be Enlisted On A Transient Basis — applications will find and use services in a small number (preferably one) of round-trips and will not require state to be held between round-trips. Neither application nor service will assume any long-term availability of the other partner in the exchange.
Services Must Support Extension And Degrade Gracefully — services must take future enhancement into account, both on the part of their own logic and exchange formats and those of other applications and services. When encountering a new version of exchange data, it should make as much use as possible of the data. Applications and services must never break if they do not receive exactly the format they expect.

Of course, these principles must observe what is readily practical, both in terms of current technology and the likelihood of heterogeneous platforms in the network. So let's consider each of these in turn. We'll consider how to go about attaining these goals and how they relate to the common threads we observed in the technologies we looked at earlier.

1. Applications Will Be Built From Course-Grained Services

We've seen that software components, while useful tools, are reusable only in applications for platforms supporting the component technology in which they were built. There is no use in trying to use a COM component on a Unix platform. (Of course, some Unix platforms have COM ports built for them, but generally speaking COM is a Windows technology.) In our case, where a development team will create just one part of an application, or where new clients are created for existing servers, we cannot count on being able to use components in a way that spans the various tiers of our applications. Instead, we will turn to services. A service is an arbitrary collection of application code that is confined to one tier and accomplishes one well-defined task. It is bigger than a component; indeed, we will frequently use components to build services for the reasons we have enumerated elsewhere. Yet, it is smaller than an application. Services model one useful part of a problem domain. If they are too small, the overhead of translating data to and from a neutral data format will place a burden on performance. If they are too big, services risk the problems of monolithic applications.

Compare our definition to that of services under Windows DNA or Windows NT. The services there are either applications that run with certain hooks into the operating system — a Windows 32 API service — or component frameworks like ActiveX Data Objects (ADO) for data retrieval. Both are bigger than a single component. ADO fails our definition only because it can rely on using a proprietary component technology across application boundaries. Apart from that, however, it would fit — it uses the interaction of several components to answer a specific problem: "Get me a result set based on a SQL query".

Often our services will model some major object in the underlying business model — customers for a sales application, or a factory line for a manufacturing application for example.

Services are usually implemented as a server-side page or small group of cooperating pages. They are characterized by a well-known data format that represents the service's object. A collection of related objects might be offered to provide a wide range of functionality. For example, a service might offer HTML in addition to its data-only formats to permit its use by ultra-thin clients such as hand-held devices. The thin client would lose the benefit of being able to manipulate the data programmatically in exchange for not having to support the overhead of the data-only formats. While it could conceivably parse and manipulate HTML, that language is marked up for presentation, not semantic meaning. It would be easier to support a different exchange format than to try to use HTML as the basis of your data markup. In short, adding HTML support, a seemingly regressive move, allows simple platforms to participate in a limited fashion provided they support a standard Web browser. Access to our services, however, will always be via open protocols. By abstracting a major object this way, we retain some of the benefits of the object-oriented model while reducing our dependency on proprietary component technologies.

Since a service is developed and maintained by a single organization, and hosted on a site under their control, we can use more proprietary technologies within the service to maximize our productivity. We expect to use component software within our services. (Often, we will be providing a wrapper for some sort of legacy software or database to avoid major rewriting efforts.) We can also utilize the full range of features of the host operating system and Web server to obtain the most value from our site. When we cross the line to another service or to a client, however, we understand we may be crossing organizational boundaries and we retreat to open standards and formats.

Note how this relates to the use of components in the future technologies we reviewed earlier. We cannot directly rely on components for the reasons we have seen, but we wish to use encapsulated bits of logic. Our applications will be built from reusable services. Within services, we rely on components. Applications use clean interfaces to reach services, and services use clean interfaces to reach the components within themselves. We are enjoying the benefits of encapsulation and delegation on both a large (service) and small (component) scale.

2. Services Will Be Discovered By Querying Directories

The network location of our services and the data formats they offer define access to our services. We must isolate these from any local conventions. While directories are still far from mature or widely deployed, we will make these the cornerstone of our service location strategy. They are pivotal to implementing location abstraction and dynamic discovery of resources. They are a resource that is useful only if it is visible to all users of the network. Consequently, we can safely assume this service will receive managerial attention at the highest levels of the information systems organization. Its structure, and the location of its servers, will be the subject of much discussion and consensus building within an organization, and it is the one area where we can safely assume global agreement and understanding within the organization.

The directory will store more than just the location of our services. It must define the capabilities of the services in terms of the business problem they address. As we shall see in later chapters, this will include a great deal of information regarding the formats in which data is offered.

We will try to define useful directory structures and tools with which to peruse our directories. Every application must be written to browse our directories in search of information about desired services, so we shall try to standardize this task as much as possible.

Although not as aggressive as Millennium, this principle directly relates to one of the common threads we saw in the three future systems. That is the thread about abstract location and dynamic discovery of data. Provided you scrupulously adhere to this principle in your own distributed systems, you can gain some of the benefits we saw in Jini and Windows DNA. Your applications will be flexible, surviving changes in the location and availability of resources. In the next chapter, I will provide you with a scheme for using the Active Directory to implement this principle. After you have made use of it a few times, it will become second nature, an organic part of your programming style.

3. Services Will Be Provided As Self-Describing Data

Why do I make so much of data exchange formats? After all, I am an adherent of object-oriented methods as they are expressed in component software, and these methods try to shield the consumer of data from its format wherever possible. The exception is needed due to the fact that we are isolating components within a service as we have defined that term.

When we hop across the platform boundaries between service pages and their clients, we must represent the object our service implements in a form readily accessible to all clients. Despite the advances in distributed object-based computing in recent years, we are still left with static data structures.

We can define the characteristics of the object in terms of metadata, and each specific object instance will be represented by a data structure that conforms to that metadata definition. Since we want exceptional flexibility and robust response to errors (see principle 5, below), we shall use self-describing data. That means that each discrete element of data is marked (tagged) with some universally recognized means of labeling the data. A consumer of the data will always know what data element it is processing (regardless of its expectations) and where the element ends because the element is tagged, beginning and end, with labels denoting what element it is. Any consumer that understands the vocabulary of the service is thus able to understand what the provider of the data is trying to communicate.

Traditionally, data is passed in a binary format where the structure of the data is shared by implication. That is, if two parties to a communication have the same message or operation in common, it is implied that they both understand the format of the data. The size, data type, and order of fields within a data structure are closely defined by design. Uncoordinated changes to data organization on the server break the client. You may have been part of a programming team working on a large client-server application in which parts of the application were assigned to different programmers. After writing the client to handle a particular data structure, you found your application broken one day. Upon investigation, you found that another programmer, responding to an enhancement request or bug report, had added a field or modified the length of an existing one without telling you. Your part of the program broke because of an arbitrary change in the data. Now, imagine the server and its structures are shared by many programming teams widely separated in geographic space and time and you can see the potential for trouble.

The use of tagged data minimizes this kind of trouble and maximizes the amount of useful sharing we can do. A consuming client or service can ignore changes it does not understand in a data structure. While we will have well-known formats, we will also tag each field within a structure. Each field is clearly delimited with the name of the field. Where fields are nested in a structure, this is made clear by the order of the delimiters. Thus, we will always be able to check for integrity on a field-by-field basis and ensure our code is responding to the data it is actually receiving, not responding to data it assumes it is receiving.

4. Services Will Be Enlisted On A Transient Basis

We must design our applications so that any persistent state is maintained solely on the tier that is interested in the computation. That is, a shopping agent is the party interested in the identity of the shopper and the shopping list, so the agent should maintain this information, not the vendor services it accesses. This works with the stateless nature of HTTP. It also allows us to minimize the dependencies between machines on our networks.

Services may come and go and clients may change their requirements. Therefore, wherever possible, a service should be written so that it need only maintain state for the duration of a single interaction with a client.

This will not always be possible, but it is the goal for which you must aim. Applications will be written as collections of clients that enlist services by making HTTP requests to obtain data. Once data is delivered, the association between the service and the client is assumed to be severed. Rather than put our efforts into ensuring state is properly maintained in a distributed system, we will devote ourselves to ensuring state need never be maintained. A client will obtain a cache of data sufficient for its needs.

From an organizational standpoint, this principle recognizes that differing development teams will change their priorities over time. An agreement reached today for maintaining state may not hold true tomorrow. This principle strives to eliminate the need for such agreements. This directly parallels the dynamic use and ad hoc release of resources we saw among the common threads of Windows DNA, Jini, and Millennium. Since we do not have the lease mechanism of Jini or the iron control of Millennium's single system image, we will limit the duration over which we must hold resources so as to minimize the chances of losing those resources while we need them.

5. Services Must Support Extension and Degrade Gracefully

Remember that one of our underlying problems is that independent development teams are at work. Time, distance, and organizational boundaries separate these teams. We have to expect some errors in the implementation of the data format. Even without errors, differing versions of the data will be common in wide area networks as new implementations are released. If each piece of data is tagged using some common convention, we can write code that checks to see what is coming next and respond appropriately.

Since different organizations have different views of common objects, we want to express our data in a way that will describe what pieces of data are held in common, and what pieces are specific to the offering organization. Similarly, data formats evolve over time, so we want to describe how the format has changed. That way, when a client accesses a service that offers data in a form slightly different from the form that the client would ideally like to see, it can still extract some information from the exchange. In later chapters, we will be introducing the Extensible Markup Language (XML), seeing how it is beneficial for our purposes, and presenting some principles for addressing the issues of format evolution and overlapping views of data. We will make use of its tagged, self-describing nature to express our problem vocabularies. Since it is written entirely in text, we can confidently assume that any computing platform we encounter will be able to read our data structures. In Chapter 4, I will introduce some conventions that allow you to express collections, version evolution, and specialization of a general format in terms of an XML document.

Summary

Cooperative network applications make life easier for practicing intranet developers who work in real organizations with real issues of management and legacy code. I've set out five ambitious principles to follow in your practice of distributed computing with Web technologies. The chapters to come will present the tools and techniques you will need to follow these principles. We've defined cooperative network applications to be those that work in harmony with good network practices. They are built with a design philosophy that promotes the flexible sharing of data, ease of administration and maintenance, and reuse of deployed server-side applications.

We set out to develop some simple principles for building cooperative applications in a dynamic, even chaotic networking environment. We took our cues from three future network technologies: Microsoft Windows DNA, Microsoft Millennium, and Sun's Jini. We examined the goals of each and the highlights of their technology. Throughout, our objective was to identify common threads that might provide insight into our task. When we were finished, we found the following points common to these technologies:

Component software
Abstract service location and dynamic discovery of location at runtime
Dynamic, ad hoc use of resources in the accomplishment of computing tasks
Formal description of desired resources at runtime
Distribution of the programming task across the network

Learning from these visions, we developed a philosophy for building cooperative applications. The limits on our philosophy are that it must work within currently available technologies and platforms, and must not require central control over development. We assume network applications are built by isolated development teams who cannot count on direct control over the code and interfaces to other applications and services. And we have defined The 5 Principles of Building Cooperative Network Applications:

Applications will be built from coarse-grained services that implement a unit of processing larger than components
Services will be discovered by querying directories
Services will be provided as self-describing data
Services will be enlisted on a transient basis
Services must support extension and degrade gracefully

The next six chapters build on these principles. Each chapter introduces a new technology or approach to implementing our philosophy. After that, we'll put these principles into practice as we build some applications in the course of three hypothetical case studies. Along the way we'll develop a few utilities you might find useful as the starting point for a personal tool set for developing cooperative network applications.

We at Microsoft Corporation hope that the information in this work is valuable to you. Your use of the information contained in this work, however, is at your sole risk. All information in this work is provided "as -is", without any warranty, whether express or implied, of its accuracy, completeness, fitness for a particular purpose, title or non-infringement, and none of the third-party products or information mentioned in the work are authored, recommended, supported or guaranteed by Microsoft Corporation. Microsoft Corporation shall not be liable for any damages you may sustain by using this information, whether direct, indirect, special, incidental or consequential, even if it has been advised of the possibility of such damages. All prices for products mentioned in this document are subject to change without notice.

Chapter 1 - Learning The 5 Principles of Cooperative Network Application Development

On This Page

Looking at Future Visions

Windows DNA

Millennium

Jini

Back to Reality

Where Are Our Present Weaknesses?

Dealing With Increased Complexity

So, What Can We Learn From The Future?

Common Threads

What is Practical Today

Where and How We Can Use Component Software

The 5 Principles of Building Cooperative Network Applications

1. Applications Will Be Built From Course-Grained Services

2. Services Will Be Discovered By Querying Directories

3. Services Will Be Provided As Self-Describing Data

4. Services Will Be Enlisted On A Transient Basis

5. Services Must Support Extension and Degrade Gracefully

Summary

Additional resources