Adding work-flow support will become critical to legacy transaction-processing applications
Meichun Hsu and Mike Howard
There are many reasons why a company would want to link complex, mainframe-resident legacy TP (transaction processing) systems with a work-flow system. TP systems are sharply focused on processing structured data for a particular purpose, such as entering a customer's order or accepting a bank deposit. Such systems, many of which are old enough to vote, are accurate and reliable, but they have certain shortcomings in today's world of inexpensive desktop processing cycles and even less expensive storage.
Moreover, most companies are changing the way they do business--pushing decision-making authority farther down the hierarchy, as middle management position
s are eliminated; making work more interesting by assigning multiple responsibilities to teams of people, each of whom performs any or all tasks that the workgroup handles collectively. Companies that are embracing globalization often establish ad hoc project teams that are geographically dispersed so that work can proceed around the clock. Semistructured or unstructured information is no longer considered unfit for business use; companies expect that systems will make allowances for data that is less than well organized. Developers--and, increasingly, end users--need an enterprise-wide view of the business processes that keep things humming.
Linking Work Flow to TP
For more than a generation, legacy TP systems have done a superior job of handling the transactions that result from discrete business events. Typically, one event triggers a cascade of others, each carried out by an application that calls, and is called by, other applications. By one estimate, a typical large bank may have from 2000 to
20,000 interconnected applications. However, many of these connections represent business policies and practices that may or may not be relevant to today's way of doing business. Often, no one can evaluate these practices--let alone modify or maintain the interconnected applications--because the linkage among related applications is encoded in the applications themselves or documented in operational manuals. Such manuals tend to become part of the oral history of the company, with all the invention, omission, and potential for inaccuracy that verbal transmission implies.
Unraveling the mysteries of current computer-enforced policies and practices is the result of business-process analysis, redesign, and modeling. This is painstaking work that involves inspecting both manual and computer-mediated procedures; identifying those procedures that are unnecessary, redundant, or outdated; and supplanting them with processes that meet today's needs and are flexible enough to be modified as conditions change. Ac
cording to a 1994 Gartner Group paper, case studies show that companies can reap more than 50 percent of the benefits of business-process reengineering even before they apply new technology to the redesigned processes. But the rest of the payoff depends on automating these processes, and that is the purpose of developing automated work-flow systems.
Work Flow as Integrator
A key role for work-flow management software will be as the integrator of disparate applications--legacy systems from the mainframe days. Indeed, more than 70 percent of reengineering efforts based on IT (information technology) will automate well-understood business processes that are currently paperbound, suffering from outdated or flawed IT systems designs, or just never connected to other important business systems. Order-entry and purchasing systems are prime examples of host-based systems that can benefit from the superimposition of work-flow systems that add processing logic and data to important existing applications and m
anual processes. It is foolhardy to believe that existing, useful business systems can be scuttled. To do so involves making enormous economical, political, and sociocultural commitments.
Increasingly, as legacy applications become modularized into encapsulated objects, the target audience for work-flow systems will be professionals and knowledge workers, who typically rely on paper, telephones, desktop applications, and perhaps E-mail. Work-flow systems will let these end users hold more power and work with more flexibility by combining application components in new ways for ad hoc response to key business events.
In the immediate future, most work-flow applications will be custom-developed and proprietary to the businesses that use them. In the near future, however, you will see the entry of work-flow system vendors, many of them alumni of the document-imaging marketplace, where they have already cut their teeth on a subset of work-flow management systems. Their products may well take the form
of work-flow templates, into which developers and end users will plug specific objects, either encapsulated from legacy systems or developed specifically to meet newly identified business needs, to accomplish specific purposes. Vendors will also differentiate themselves by offering prefabricated work objects. Furthermore, most of the major platform vendors in the marketplace today are building large direct-service capabilities to provide systems integration, not necessarily biased in favor of their own products.
Linking work-flow management and legacy host-based systems presents challenge and opportunity to independent software developers and corporate IT organizations alike. IT organizations can give business-process reengineering efforts a real-world flavor by exploring ways to incorporate existing applications as work units in an open and flexible work-flow management system. Rather than letting boardroom demands drive business-process reengineering, IT can establish itself as a leader in solving b
usiness problems by developing a structured and informed approach to work management and related systems. In addition, following the example of American Airlines, whose information-systems business unit sold CASE templates developed for in-house use to other airlines, IT shops may discover profit-center potential in their own work-flow management frameworks.
Defining Work Flow
Today's work-flow systems are largely homegrown; most are currently embedded in or integrated with document management systems--for example, loan application processing systems (see the figure "Execution of a Simple Flow"). More highly evolved work-flow systems treat document images as objects and route them through office mail or messaging systems, allowing notification of the end user when work arrives at the workstation. This is all very useful, but the true payoff of work-flow systems will be realized when the abstractions of business processes can be taken to a higher level, where business users can understand and manipul
ate them to optimize processes for competitive advantage. This requires that work-flow management systems be separated from document management or office systems and that they serve as an umbrella for such systems.
A full-fledged work-flow management system is one that supports the development, execution, and analysis of multistep, multiuser business processes. It manages the flow of work among participants in the system, based on business-specific procedures, constraints, and objectives developed during the process-analysis and modeling efforts. Flow descriptions--not to be confused with data flows, which deal with the movement of data, not work--are the programs that work-flow systems follow in implementing a business process. A flow is a compound work unit, composed of modular work units called steps, and possibly other nested flows. Each work unit in the flow has its own description. Modular work units are combined according to dependencies among them. Event-driven triggers can cause the modular wo
rk units to execute sequentially or in parallel. Flow descriptions and associated process information are stored in a process repository, just as metadata--information about the data with which an application works--is stored in a data dictionary.
A work unit is an application routine (or a set of routines, if the work unit is a nested flow); each work unit is necessary for accomplishing a single business function. Applications can use different programming paradigms and run on different platforms; they are interrelated by information and control flow. A flow can operate within or across organizational boundaries.
The work-flow system ensures that individual flows execute reliably and continually until they are properly terminated. The system tracks execution events and supports user inquiry and analysis of the flow's current execution status and history. Exception-handling actions, such as retry and compensation, may be programmed in the flow description; alternatively, they may be specified ad
hoc by control commands during execution. Exception handling may also require synchronization with applications.
Work-Flow Life Cycle
The life cycle of the work-flow system is similar to that of other application systems (see the figure "Life Cycle of a Work-Flow Application"), recursively moving through the phases of analysis, development, execution, and administration. Each phase requires its own environment. Similarly, those who interact with work-flow systems fall into four categories. Business-process analysts analyze business operations during the analysis phase, using BPR (business-process reengineering) tools and business-process models. They identify objects and their behaviors (or methods) in an abstract manner and provide input to developers. They can use feedback from the execution phase for continuous process improvement.
Developers design and capture flow and applications descriptions during the development phase. Flow developers develop flow descriptions, using flow developmen
t environment tools such as a flow editor and a flow simulator. Applications developers develop the applications that form the bases of steps, using applications development environment tools.
Workers, or users, initiate and carry out the execution phase. These are the people who perform the work in a step, on whose behalf servers perform work, and who handle exceptions when they occur.
Administrators manage the business process that the work-flow system addresses through most of the life-cycle phases. They manage organizational process databases that hold resource descriptions and policies, such as security and resource utilization policies. They may handle flow exceptions, monitor and control execution, alter flow descriptions to react to special situations, and update descriptions. They are different from systems managers, whose tasks fall outside this discussion, in that they focus on business processes and not on the technical aspects of the computer system itself.
Principles of Work-
Flow Architecture
Certain principles provide the framework for a work-flow management system architecture. First is the separation of flow control and data management. The work-flow component manages complex events, triggers, and organizational policies and constraints as specified in high-level descriptions. It cooperates with data managers but is not embedded in them. A second principle holds that the work-flow system is composed of open components that are flexible, customizable, and replaceable. The heterogeneity principle allows multiple heterogeneous applications environments to participate in work flow. The reliability principle holds that work-flow systems can use transactional synchronization and recovery services to preserve data consistency when necessary. The principle of scalability declares that a work-flow system can be partitioned into logical areas of independent control called domains. A single flow can span multiple domains. The final principle, infrastructure integration, enables the w
ork-flow system to use standard services, such as object-request brokerage, name and transport services, and message and data-interchange protocols.
A work-flow system is not a TM (transaction manager), although it may use a transaction manager for synchronization with database managers. The TM of open TP monitors is typically a separate, open component. Also, a work-flow controller is not a TPM (TP monitor), although it must interoperate with standard, open TM services. Finally, a work-flow system is not intended for coordinating relatively unstructured groupware activities or for monitoring personal agenda and calendar applications. However, these applications and tools can interact with the work-flow system to offer complete work-flow functionality.
Managing the Evolution
The work-flow approach is applicable to legacy TP systems, as well as in other areas where support for developing and reliably executing long-lived computations is required (e.g., CAD/CAM, electronic business documents, a
nd imaging applications).
TP applications are typically developed using a TPM, which provides a complete environment for supporting integrated database-intensive applications. The TPM funnels requests and results between terminals and applications servers and provides queuing and transaction management services that ensure recoverability and optimize performance. A current trend is to downsize TPMs--to free them of their proprietary constraints and deploy them on open systems, providing client/server interfaces and accommodating open database servers and heterogeneous applications development environments.
Some proprietary TPMs' role in efficiently servicing an extremely high volume of short, specific requests against a large amount of shared data is impossible to replace anytime soon. However, you can off-load many applications that are crowded onto an aging TPM system--although not without pain--and better manage them with an open TM system. Open TP is a critical stage in modernizing TPMs, but
further developments are needed to allow next-generation TPMs to accommodate standard and extended-service components.
For example, the CORBA (Common Object Request Broker Architecture) standard, endorsed by the Object Management Group, may provide the backbone for heterogeneous applications integration. The services of the Object Request Broker allow functions (called methods in object-oriented circles) to be organized in an object-oriented framework, enhancing manageability and extensibility of the applications servers that provide them. An extensible transaction manager may support extended transaction models, such as nested transactions and sagas, and allowing new transaction models to be defined dynamically. Application and database servers in the next TPM generation may be active (i.e., they may be capable of monitoring events of interest to other components in the system and providing event notification when these events occur).
If a work-flow management system is to be effective in miss
ion-critical business-process management, then it must be able to integrate TPM-based applications into its modular work units, or step applications. The marriage of TP and work-flow technologies is a necessity to make business-process abstractions available on both the development and execution sides of the business.
This marriage will probably occur in three stages. In the first stage, the terminal-emulation capability provides the linkage to legacy TP applications on monolithic TPMs. This is happening now. At the user-system interaction level, a step in a work flow appears as an application that a conventional TPM application expects. The work-flow system uses terminal emulation to interact with the TP system, activating TP applications and sending information to and extracting information from the legacy system. At this stage, there is no redevelopment or reengineering of the legacy system: Integration may be laborious, and there is a reliability window with respect to the flow state.
In the
second stage, TPM applications are developed with a client/server model. APIs supplant terminal-level interfaces, allowing a step application to invoke the APIs while presenting to the end user a modern user interface. In addition, a step application can leverage services provided by other IT components, such as document and imaging systems, and desktop utilities and tools. This stage requires the existence of both open TP monitors and open work-flow managers--in the sense that appropriately documented APIs are available.
The third stage features an integrated applications development life cycle, in which business-process analysis and reengineering tools, as well as organization resource managers, such as calendar applications and corporate resource-policy databases, are able to work together with work-flow managers and TPMs. At this point, front-end business-process analysis tools produce business-process templates, from which work-flow managers automatically or semiautomatically synthesize and execu
te work flows.
Work Flow Ahead
The advent of the complete work-flow management environment is not far away. According to a 1993 Gartner Group study of the consultancy's clients, 67 percent currently use E-mail and 90 percent expect to have it in use by 1996. Calendar and scheduling applications are installed in 33 percent of companies, with 63 percent expecting to have them on-line by 1996. Business-intelligence systems, used for near-real-time analysis of marketing, point-of-sale, and other types of data for strategic decision-making, exist in 12 percent of companies today, with 34 percent intending to have them installed by 1996.
In addition, the study found that imaging, document, and work management technologies will reach critical mass by 1995 and will then become fundamental building blocks in the electronic workplace of the future. The broad use, consolidation, and maturation of these and other applications, in conjunction with continued pressure to supplement and support existing lega
cy systems with desktop systems, will ensure an increasingly critical role for work flow--and for the analytical and modeling tools that support the design, development, and deployment of work flow on legacy systems.
Figure: Execution of a Simple Flow
Execution of a simple flow in a loan-processing system proceeds as follows:
1. The flow controller notifies user A (through user A's worklist manager) that step 1 is waiting to be done.
2. User A chooses to perform step 1.
3. The work-flow system's client environment invokes the application needed to perform step 1.
4. The application interacts with the flow controller and provides the application's initial arguments and then collects its results.
5. When step 1 is finished, the flow controller routes the result to the next step and notifies user B that step 2 is waiting.
6. The sequence continues for steps 2 and 3.
As the loan flow progresses, the flow controller writes execution events into the history repository, from which the use
r can retrieve information relating to the progression of the flow.
Figure: Life Cycle of a Work-Flow Application
Phases and associated tasks are as follows:
Analysis--Analyze the targeted departmental or enterprise-wide operations. Identify resources, data objects, policies, and procedures.
Development--Using results of the analysis phase, construct executable work-flow descriptions. Populate organizational databases, develop applications and their data-object managers, or integrate with existing applications and data managers.
Execution--Create and execute work-flow instances.
Administration--Using the status and history information provided by the execution phase, analyze history data and provide feedback to the analysis and development phases.
Meichun Hsu is DEC's corporate architect for work-flow systems. She works for the Activity Management Group in Palo Alto, California. You can reach her on the Internet at
mhs
u@pa.dec.com
or on BIX at
editors@bix.com
. Mike Howard is vice president and service director of office information systems at the Gartner Group in Stamford, Connecticut. You can reach him on the Internet or BIX at
editors@bix.com
.
Work-Flow and Legacy Systems
