varied application needs now and through the millennium. Yet without a clear definition of terms, each new application project will result in the purchase of a new middleware platform. The consequence will be a variety of redundant, incomplete, and even contradictory products.
Most organizations already have a middleware muddle on their hands. Having purchased SQL middleware from database vendors in the early 1990s, many found disappointment in the limited scalability, performance, and range of functions these products offered. Organizations are repeating this scenario by using new development tools (e.g., Forté and Open Environment's Entera) without proper evaluation of the middleware those tools incorporate.
The changing nature of middleware compounds th
is confusion. Middleware products are evolving from specialized system-level facilities to broad-based environments supporting many types of applications. Those low-level interfaces and protocols that only rocket-scientist network programmers could master are now turning into application-level products that mere mortals can navigate successfully. Middleware has evolved from products such as IBM's APPC, an interface to the LU6.2 protocol, to today's application-level products. Now, middleware is evolving from products that perform only one function--such as remote SQL access--to those capable of many functions.
Defining Middleware
Can't define middleware, but you know it when you see it? Pop-quiz time. Is Information Builders' EDA/SQL middleware? OK, that one's easy. What about Novell NetWare? Hmm. How about the Internet's World Wide Web? Is that middleware? Well, maybe a definition wouldn't hurt.
Middleware is software that allows elements of applications to interoperate across
network links, despite differences in underlying communications protocols, system architectures, OSes, databases, and other application services.
The key parts of this definition are "elements of applications," "interoperate," and "despite differences." First, middleware isn't meant specifically to link physical clients and servers (that's the job of connectivity protocols such as TCP/IP). Rather, middleware seeks to link the logical elements within applications. Second, the paramount goal of middleware is any-to-any interoperability. Application modules interoperate using a variety of methods, including file exchanges and sharing, shared databases, transactions, and remote procedure calls (RPCs). Third, middleware shields applications from diversity in the underlying environment. Middleware helps your applications operate happily in any environment.
We can recognize EDA/SQL in this definition. We also recognize messaging products such as IBM's MQSeries transactional middleware. However, using
this definition, we can also answer the question about NetWare. It is, in a way, middleware. NetWare can link logical application elements. It does so by supporting network file operations. Novell's new Net2000 API initiative will also support database operations, transactions, and other modes of interaction. However, NetWare is still hardware-dependent, so it fails the test about overcoming differences in underlying system architectures.
We can, however, conclude that the Web is middleware. The Web isn't a private network technology--at least not yet. But it provides the means to link application elements, supports file-based interoperability, and is independent of underlying platforms. That's one of its most popular features and one reason some think of the Web as the ultimate in middleware.
Our definition will help us identify middleware, but identifying is not enough. After all, EDA/SQL and the Web have similarities in intent and function but many differences in structure and intent. To be a
ble to select the appropriate products, we need to dive a bit deeper into our definition of middleware and determine what specific features distinguish one middleware product from another.
The Components of Middleware
All middleware products have the same basic components. To understand those components, we'll tack on a sentence to the end of the basic definition of middleware that was introduced earlier: Middleware achieves its purposes by providing application-level protocols and formats, access to application services, support for one or more application models, and administrative facilities (see the figure
"Components of Middleware"
).
Middleware products address all three phases of applications development: development, execution, and deployment (including management). Of course, most of us are interested in the execution environment. Over the last several years, middleware has expanded in this area.
In the early days, middleware was almost tota
lly communications software. For example, PeerLogic's Pipes platform, Intersolv's recently acquired SequeLink, and Software AG's Entire Broker are almost pure communications products. Now, however, the execution environment includes application services (e.g., transaction monitors, SQL optimization and routing, and database replication). The trend during the last year has been toward bundling: Vendors bundle communications and other products and integrate them with application services.
Communications Services
Still, middleware starts with communications services. The communications protocols and formats of middleware describe interactions among application components. By contrast, TCP/IP, Novell's IPX/SPX, and IBM's Systems Network Architecture (SNA) protocols are network transport protocols, not to be confused with middleware. Middleware communications services do use these and other networking protocols. Middleware provides three basic communications services: formats and protocols
, protocol services, and control services.
A format describes the structure of a message that will travel across the wire, including the syntax required to create that structure. A protocol defines the on-the-wire representation of the message. An RPC's format and protocol are different from a remote SQL access product's format and protocol. The trend in middleware is toward supporting more than one protocol and format within the same product.
Protocol services add some useful features to basic communications. These can include marshaling and unmarshaling of messages, platform data-format translations (e.g., big-endian and little-endian), message compression, transport protocol translation, and message encryption.
Not all middleware products provide the same services. Marshaling and unmarshaling is a fundamental service, but the other ones are optional. Still, some of the protocol services, like compression or encryption, may be vital to your operation.
Control services support one
or more communications model or style of distributed processing. Each model differs in the way it structures communications among the elements of applications. Control services provide the signaling protocols, queues, message binding, and other services required to support each communications model.
The 10 Communications Models
There are 10 communications models that middleware control services support: datagram, one-shot, query, asymmetric, and symmetric models, each in network-dependent and network-independent flavors (
see the table
). A key characteristic of each model is the need for an immediately available network link. Network-independent models have message-queuing facilities that effectively allow postponing the completion of an interaction if no link is available. Benefits can include sharing of network resources and deferral of communications until low-cost hours.
Each communications model specifies a style of interaction between two (or more)
application elements. For example, the datagram is one-way: An application sends out a message without expecting any response. This model is useful for applications that monitor devices, systems, and other applications.
By contrast, in the one-shot model, an application issues a single request message and expects a single message back. For example, a formula and some values might go to a fast calculation server, and the result would return.
The query model is similar to the one-shot model, except that the sender expects more than one response. For example, a database query triggers a chain of messages bearing the results set.
The asymmetric and symmetric models can raise the throughput of messages by allowing applications to send and receive as swiftly as bandwidth availability permits. In asymmetric models, incoming and outgoing messages share a single communications channel, so the module can send or receive but not both at the same time. With symmetric models, incoming and outgoing me
ssages have their own channels. Both models must track the context of a given interaction within a flurry of messages.
What about synchronous and asynchronous communications? You should forget about them. At one time, they represented a real distinction among middleware products. However, in recent years, vendors of so-called synchronous technologies have added asynchronous capabilities to their products. Also, vendors of asynchronous products have added support for synchronous modes of communication. Therefore, this distinction has for the most part disappeared.
Evaluating middleware using the 10 communications models listed here is more productive.
What's Your Name and Who Are You?
The execution environment of middleware also includes core services. These support the operation of middleware products in several ways.
The most important core services are logical name management and security. Name-management services, often called
directories
, map the logi
cal names of application modules to their physical addresses in the environment. Security services control access to application modules. For example, the Open Software Foundation's (OSF) Distributed Computing Environment (DCE) contains naming and directory services and a security service.
Other core services include distributed memory management, gateways to external systems, and server-to-server communications services that can segment large environments into domains.
The trend is to let the user choose those core services needed rather than bundle them with a product. For example, Open Horizon's Connection lets the user choose from among several security and directory products. So does Entire Broker. Both products use available APIs to directory and security to give the user a choice.
Applications with Personality
The execution environment also includes application services that support the guts of application functions, such as SQL database access, transaction proces
sing, E-mail distribution, work flow, and document management. Application services give middleware a personality stamp that users recognize (e.g., submitting a customer order or confirming inventory).
Middleware today offers 10 such personalities, which we call
application models
(
see the table
). The application models appear in order from the most concrete, at the top, to the most abstract, at the bottom.
Store-and-forward and publish-and-subscribe are models that users will probably recognize. Store-and-forward is the basis for E-mail. Publish-and-subscribe is an up-and-coming model used in systems integration. The two models are related. The difference: In the subscribe half of publish-and-subscribe, application modules "pull" data in from other modules. In store-and-forward (and the publish half of publish-and-subscribe), modules solely push data out to the environment.
Work flow is also a model that users readily recognize. Work flow employs conditio
ns--and satisfaction of conditions--to route information and responsibility from one application to another.
Distributed transactions handle a database update using the customary two-phase commit protocol to complete the task.
Remote file access is a familiar task, something users do every day. The same is becoming true of remote database access, though many users have little experience with it. Data warehousing will introduce remote database access to many users.
The last five models are probably all mysterious to the average user but are familiar--and helpful--to applications developers. Distributed object interaction and access to remote functions are closely related. While the developer employs an object-oriented design with distributed objects, he or she uses good old subroutines with remote function access .
Distributed database management and database replication are both used for synchronizing distributed data. However, in distributed database management, the developer crea
tes a single logical view of distributed databases. In database replication, the developer synchronizes multiple copies of one database.
In the distribution-of-display model, the developer employs a terminal data stream to create a remote representation of a host-based application's presentation functions. It's just like being there.
What? No messaging? Nope: We don't include it because this term (like synchronous and asynchronous) has become too vague to be useful in distinguishing middleware products.
In a "one from column A, one from column B" style, messaging products generally implement the store-and-forward application model using the one-shot/net-work-dependent communications model. By using the 10 application models and the 10 communications models in this way, you can reveal any middleware product's true nature.
Middleware support for applications development ranges from full-blown development environments to naked APIs. That support will determine how transparent distribu
ted computing will be to applications developers. Usually, APIs will force developers to master message binding, memory management, and other mechanics of distributed processing. Interface definition languages (IDLs) hide some of these mechanics. Object request brokers (ORBs) such as IBM's SOMobjects, Sun's NEO, and Iona's Orbix automate message binding. IDLs conceal the underlying mechanics of distributed processing, not the fact that application components are on a network.
Multi in the Middle
Until recently, a middleware product would offer just one application model. DCE gave users the remote function access model, but not remote SQL access. Oracle's SQL*Net supported remote database access, but not database replication or distribution of display. With single-function middleware, each new application might need a new middleware product. The result: middleware hell for corporate technology planners. Multiple products mean high support costs and low leverage of technology investment
s.
This began to change about two years ago, with the introduction of multifunction middleware products. The first was the TransAccess product line from Netwise. Netwise was a leading RPC environment vendor and specialized in linking LANs and PCs to CICS and mainframe COBOL programs. In 1994, Netwise added a SQL-access product built on top of its RPC. The result: support for two of the 10 application models in the same middleware environment.
More recently, the EDA/SQL product line has added support for the OSF DCE RPC and API. Sybase is porting its OpenClient remote database access API to its new store-and-forward middleware environment (EMA).
The trend toward multifunction middleware will continue. Software vendors will start competing on combinations of functions. Multifunction middleware is good news for user organizations because it promotes leverage. Now, users can benefit from a single middleware environment that supports a variety of application models.
Don't Diddle wi
th the Middle
Another factor that may be contributing to the confusion about middleware is the specialization inherent in the very structure of corporate IS organizations: Database administrators think about SQL. Developers concentrate on tools. Deployment specialists work with network and administration systems. Yet middleware cuts across all these specialties, demanding new coordination of specialists.
To foster this cooperation, IS organizations must find a common ground for their specialists on middleware. Agreeing on terminology is helpful. Examining the specific components of middleware ensures that consideration and comparison of new products will be rational and based on fact, not vagueness.
WHERE TO FIND
Forté Software, Inc.
Oakland, CA
Phone: (510) 869-3400
Fax: (510) 834-1508
IBM
Armonk, NY
Phone: (800) 426-3333 or (914) 765-1900
Information Builders, Inc.
New York, NY
Phone: (800) 969-4636 or (212) 736-4433
Fax: (212) 268-7470
Intersolv, Inc.
Rockville, MD
Phone: (800) 547-4000 or (301) 838-5000
Fax: (301) 231-7813
IONA Technologies, Inc.
Marlborough, MA
Phone: (800) 672-4948 or (508) 460-6868
Fax: (508) 460-6099
Netwise, Inc.
Boulder, CO
Phone: (800) 733-7722 or (303) 442-8280
Fax: (303) 442-3798
Open Environment Corp.
Boston, MA
Phone: (800) 562-5969 or (617) 562-0900
Fax: (617) 562-0038
PeerLogic, Inc.
San Francisco, CA
Phone: (800) 733-7601 or (415) 626-4545
Fax: (415) 626-4710
Software AG of North America, Inc.
Reston, VA
Phone: (800) 423-2227 or (703) 860-5050
Fax: (703) 391-6731
Ten communications models that middleware supports, with sample
applications for each.
COMMUNICATIONS NETWORK-DEPENDENT NETWORK-INDEPENDENT
MODELS EXAMPLES EXAMPLES
Datagram
Single outbound Network device Paging system
message status notification
One-shot
Single outbound Network-based File transfer
message; single calculation server
response
Query
Single outbound LAN-based database Web search
message; chained query
responses
Asymmetric
Multiple outbound Customer-service Web-based publishing
messages and call center application
responses; single
session
Symmetric
Multiple outbound High-throughput High-throughput Web
messages and reservation system publishing application
responses; in two
dedicated
channels
The 10 application models of middleware, arranged from the most
concrete (at the top) to the most abstract, from a user perspective.
1 Store-and-forward/publish-and-subscribe
Directs message to recipient either by pushing data out
(store-and-forward and publish) or by pulling it in (subscribe).
2 Work flow
Directs message to recipients according to either conditions or
policies.
3 Distributed transactions
Manages simultaneous updates to multiple databases under transaction
control.
4 Remote file access
Redirects file requests across a network.
5 Remote database access
Transmits SQL requests across network to servers.
6 Distributed object interaction
Supports messaging between objects across a network.
7 Access to remote functions
Redirects program call to a function across a network.
8 Distributed database management
Maintains a single local database across multiple physical databases.
9 Database replication
Synchronizes copies of a single database.
10 Distribution of display
Makes presentation functions of an application available to a remote
client.
illustration_link (7 Kbytes)
Middleware was once communications so
ftware only. However, today's middleware includes development tools, an execution environment, and deployment facilities.
Of these three components, the execution environment is typically the primary thrust of middleware. Deployment facilities and development tools are often either rudimentary or absent.
John R. Rymer is vice president and senior analyst at Giga Information Group (Cambridge, MA). You can reach him on-line at
jrymer@gigasd.com
.
The Muddle in the Middle
