Inter-task co-ordination in long-lived distributed applications

Abstract

We will discuss issues in the design of a fault-tolerant application composition and execution environment for distributed applications whose executions could span arbitrarily large durations. We are particularly interested in the domain of electronic commerce applications in the Internet/Web environment. The Internet frequently suffers from failures which can affect both the performance and consistency of applications run over it. A number of factors need to be taken into account in order to make these applications fault-tolerant. First, most such applications are rarely built from scratch; rather they are constructed by composing them out of existing applications and protocols. It should therefore be possible to compose an application out of component applications in a uniform manner, irrespective of the languages in which the component applications have been written and the operating systems of the host platforms. Application composition however must take into account individual site autonomy and privacy requirements. Second, the resulting applications can be very complex in structure, containing many temporal and data-flow dependencies between their constituent applications. However, constituent applications must be scheduled to run respecting these dependencies, despite the possibility of intervening processor and network failures. Third, the execution of such an application may take a long time to complete, and may contain long periods of inactivity (minutes, hours, days, weeks etc.), often due to the constituent applications requiring user interactions. It should be possible therefore to reconfigure an application dynamically because, for example, machines may fail, services may be moved or withdrawn and user requirements may change. Fourth, facilities are required for examining the application's execution history (e.g., to be able to settle disputes). So, a durable ‘audit trail’ recording the interactions between component applications needs to be maintained. Taken together, these are challenging requirements to meet!

Our approach has been to implement the application composition and execution environment as a transactional workflow system that enables sets of inter-related tasks to be carried out and supervised in a dependable manner [1,2]. Workflows are rule based management software that direct, co-ordinate and monitor execution of tasks arranged to form workflow applications representing business processes. Tasks (activities) are application specific units of work. A Workflow schema (workflow script) is used explicitly to represent the structure of an application in terms of tasks and temporal dependencies between tasks. An application is executed by instantiating the corresponding workflow schema.

We will describe how our system provides a uniform way of composing a complex task out of transactional and non-transactional tasks. This has been made possible because the system supports a simple yet powerful task model permitting a task to perform application specific input selection (e.g., obtain a given input from one of several sources) and terminate in one of several outcomes, producing distinct outputs. The system has been structured to provide dependability at application level and system level. Support for application level dependability has been provided through flexible task composition mentioned above that enables an application builder to incorporate alternative tasks, compensating tasks, replacement tasks etc., within an application to deal with a variety of exceptional situations. The system provides support for system level dependability by recording inter-task dependencies in transactional shared objects and by using transactions to implement the delivery of task outputs such that destination tasks receive their inputs despite finite number of intervening machine crashes and temporary network related failures; this also provides a durable audit trail of task interactions.

Our system architecture is decentralized and open: it has been designed and implemented as a set of CORBA services to run on top of a given ORB. Wide-spread acceptance of CORBA and Java middleware technologies make our system ideally suited to building Internet applications.