Groupware success comes when developers and end users alike understand the technical hurdles that confront the launching of applications
Kelly Trammell
Pity the poor groupware developer. It's easy for managers and end users to dream up enterprise-wide applications that foster collaboration and seamlessly tie an organization's talent into efficient workgroups. But dig deeper, as developers must, and reality bites. Issues such as immature programming tools, data-synchronization complexities, effectively handling middleware layers, sorting out mail interfaces, and accommodating a myriad of network protocols can quickly turn those dreams into multiheaded workgroup applications from hell.
Developers will grapple with most of these issues until everyone starts using the same OS, mail-transport platform, an
d network protocols. Because that's not likely to happen anytime soon, organizations need to understand the key technical hurdles that confront the building and implementation of groupware applications. This knowledge will give managers and end users more realistic ideas about what types of applications are possible and practical. At the same time, developers will be able to write more efficient applications, and they'll get them up and running faster.
Programming Limits
The first problem that groupware programmers face comes from development tools that often cater more to end users than to hard-core developers. For example, the Notes development environment is still loosely based on the Lotus 1-2-3 macro language from the early 1980s.
In this environment, the Notes
@commands
and
@functions
are the primary programming tools. These are macro commands that execute options from the Notes menus, call external programs, and run common algorithms and calculat
ions. Any type of coding task that is not included as an @function (e.g., sequential number generation) becomes a long and complex macro-based script or requires a Notes API call. Both of them can be tedious to code and difficult to test and debug.
Unlike a procedural language, Notes cannot perform DO loops. Notes also lacks tools such as an integrated debugger, version control, and a report writer. As the rest of the development world moves toward class libraries and OOP (object-oriented programming) techniques, Notes offers developers more modest tools: common, reusable templates for building generic applications, such as a discussion database or forms-processing system.
As a result, developers face an interesting dilemma. They can use macro-based programming tools in Notes and rely on API calls to handle tasks that go beyond the capabilities of an @function. Or they can program the entire application in C++ and write a custom mail interface, multiplatform client, middleware layer, and data-sy
nchronization code.
In the interests of time, budget, and user satisfaction, many programmers choose Notes. Most developers would rather give up control on the development side than write a complicated, specialized data-synchronization or replication routine.
Another important limitation of the Notes development environment is the way it restricts programmer and end-user interaction to forms for data input and views for displaying data. Items such as custom controls, ad hoc queries, event trapping, charts and graphs, and customized reports don't exist in this environment. To get around this, developers use low-level, custom DLLs and the Notes API.
Since last fall, several products, including Powersoft's PowerBuilder Library for Notes, Lotus's HiTest Tools for Visual Basic, and Revelation Technologies' OpenInsight, have appeared. They link event-oriented tools for connecting to and manipulating Notes data. They give developers control over the application, user interface, and data store to
help them work around Notes limitations. Developers can build simple or complex custom front ends to Notes and regain control over the user interface and application.
Some tools, such as the Lotus Notes ViP (Visual Programming) environment, give developers control over Notes data replication and messaging. These tools let developers build new types of Notes applications, such as executive information systems, query builders, and decision support systems that use charts and graphs.
The biggest enhancements to Notes from a developer's perspective will come when release 4.0 ships (see "Competing Platforms"). Of most interest to developers will be native support for both X.400 and SMTP mail protocols, which will make exchanging mail and documents with external mail systems easier.
Lotus Script, a visual development tool, will bring additional controls and procedural-language capabilities similar to those provided in Visual Basic. New OLE 2.0 support in Notes 4.0 will simplify interapplicatio
n integration. Finally, Notes 4.0 will support field-level data replication, to simplify data synchronization between remote clients and servers.
What to Do with Data
Workgroup applications often must receive data from multiple sources, perform some type of compiling or filtering process, and then replicate the processed information to distributed clients. Because these tasks happen periodically, synchronization and integration are critical issues. Even with new tools, APIs, and utilities available to integrate traditional data sources with workgroup tools, data integration remains a significant implementation hurdle. (For details about data replication, see the article "Replication's Fast Track".)
One of the problems is that groupware platforms, including Notes, handle data differently than do traditional mainframe- or SQL-based DBMSes. Notes collects and stores data in an unstructured format that is good for group collaboration and coordination but problematic for tran
saction processing, querying, and reporting. Therefore, integration with these types of systems often becomes the linchpin in making a workgroup system work.
For developers, Notes provides some simple tools for importing and exporting data, but they are limited to transfers of structured or tabular text from spreadsheets and single-user databases. The next step up for developers is to write a C program that calls the Notes API to move data in and out of Notes databases. This is a good solution for solving specific problems, such as a one-time data migration or initial database load. However, this solution levies a high maintenance cost: You can rarely reuse these interfaces, and you must modify them whenever you change either side of the application or when the source of the data is different.
Another option is to use one of the new data-interchange tools that are available for Notes and SQL databases (see the sidebar "Strained Relations"). These middleware products offer developers point-and-cl
ick mapping from source to target data and insulate the developer from having to work the underlying plumbing that is moving the data. These solutions work well as long as the source data you need is timely, stored in the correct format, and at the level of detail required by the target application.
Until recently, middleware products were server-based and expensive. They needed dedicated server hardware, were priced on a per-server basis, and required a great deal of custom programming. These products are generally limited to scheduled or batch migration and cannot support ad hoc queries or dynamic updates.
However, newer middleware tools such as Trinzic's InfoBroker provide dynamic access to foreign data sources, such as Oracle, SQL Server, or DB2. InfoBroker can run as an add-in task on the Notes server.
Better tools won't solve all your replication problems. Performance remains an overriding concern in any replication system you create. For example: A company wanted a Notes-based proj
ect management application to pull project expenditures from a mainframe general ledger daily and then replicate the information to project managers in the field.
However, the general ledger could not report actual expenditures by project, and the general-ledger information was updated only weekly. The solution came in bypassing the general ledger and writing a query routine on the mainframe to extract and sort the expenditure data by project from a transaction file. The application then imported this extract file into the Notes database every night for distribution the next day to project managers throughout the company.
This worked because the amount of data being moved was way below 1 GB, which is technically the size limit for Notes 3.x databases. The practical limit from a performance standpoint is closer to 300 to 400 MB, depending on the number of users, the number of forms and views, and the complexity of the application. A query that pulls down 500 MB or more of data into a Notes databa
se is like pointing a fire hose into a paper cup.
Mail Sort
Different E-mail systems and network protocols turn developers into systems integrators to build applications that work for everyone. Interoperability often requires developers to find lowest-common-denominator standards, which can limit an application's robustness and performance. It is common for organizations to have multiple, disconnected E-mail systems. Some companies may have 10 or more, although the long-term goal is to move to no more than three E-mail systems.
If you are writing an application that calls for mail integration, you must decide what level of integration is necessary. Mail integration can be as simple as converting mail messages to text and importing them into a common data store. Alternately, the integration approach taken can be as complex as initiating a work flow based on keyword triggers within the mail message.
A developer must figure out which mail standard and protocol to use
. An alphabet soup of competing messaging standards and APIs exists, including MAPI, VIM (Vendor-Independent Messaging), MHS, X.400, and CMC (Common Mail Calls). Notes and cc:Mail support VIM. Using it, developers can write an application so that users forward cc:Mail messages to a Lotus Notes database, where the message text appends to a Notes form and initiates a work-flow process.
To perform the same tasks with other mail systems, such as MHS or MAPI, developers would run the message through a mail gateway for conversion to a workable format. Gateways are typically stand-alone processors that convert foreign addresses and message text to a common format for delivery by host mail systems.
Like any other conversion, this process is an inexact science. Mail systems have unique features and functions that other systems may not support. Developers must determine what trade-offs are being made by the gateway and determine the impact on the application.
Some common problems with mail gateways
include message text becoming truncated or address and subject fields getting garbled or lost. In an application where the subject field routes a product-request form to the proper department, the gateway conversion must be clean and reliable. Some organizations continue to operate proprietary mail systems for which no gateway exists. If the application requires a mail interface, the developer usually has to write the conversion code as part of a front end to the workgroup application. The worst-case solution is for the application to convert the data to ASCII, parse the data to figure out the addressing and message text information, and then import the data into the target database.
Remember the End User
A groupware application can't be successful unless it's easy to use and scalable. Part of the design work involves meeting with end users to determine how they will access the system. Some people may connect into a central server; others use a LAN connection or a remote-acces
s server. Developers need to determine how the application will handle each access method.
Today, it is also common for workgroup applications (e.g., help desks and customer service) to support a myriad of requests or questions from E-mail, fax, telephone, and wireless devices. By comparison, COBOL programmers rarely have to worry about segmenting the user population by access method or the layout of the enterprise network.
Each access type has some issues associated with it--capacity and bandwidth, compatibility, and security--that have to be sorted out as part of the application's design task. For example, it certainly doesn't make sense for 15,000 salespeople to dial into one or two remote-access servers in an evening.
A similar problem with capacity exists when remote users or servers are replicating Notes documents that have large file attachments or embedded images. Moving documents that are 2 MB or larger over a dial-up connection can take up 8 hours.
Scalability becomes an
issue when the infrastructure for a groupware application was initially built for small pilot groups. Few organizations prepare an enterprise architecture plan to support current and future applications requirements. As a result, enterprise architectures that are planned around 100 users in three major cities have a difficult time scaling to 1000 users on three continents. Because the application drives infrastructure requirements, the developer often carries out the task to plan and build the infrastructure, which is often a larger project than writing the application itself.
Another common implementation problem is that not everyone in the workgroup may have the technology needed to run the application. The first question to ask when you get a request for a multiuser, multilocation application is whether all participants are on some common system (e.g., Notes, mail, or forms). The usual answer is, "Not yet, but probably by the end of the year."
Faced with that response, do you roll out a group
ware product such as Notes before you develop your applications? If you do, you risk early user rejection because of no immediate benefit. Or do you develop essential applications and then deploy Notes? The time delays minimize your application's return on investment and eats away at the shelf life of your solution.
An effective answer is to split resources into a development team and a Notes deployment team. You should allow the developers to stay one step ahead of the deployers so that applications requirements can be fed to the deployers on a just-in-time basis.
As groupware matures, the issues outlined above should diminish with new workgroup tools and standards for interoperability. Until then, workgroup computing will remain a challenging but fertile area for developers.
WHERE TO FIND
Big Sky Technologies
San Diego, CA
(800) 488-4188
(619) 496-2100
fax: (619) 496-2119
Lotus Develop
ment Corp.
Cambridge, MA
(800) 346-1305
(617) 577-8500
fax: (617) 253-9150
Powersoft Corp.
Concord, MA
(800) 395-3525
(508) 287-1500
fax: (508) 287-1600
Revelation Technologies
Stamford, CT
(800) 262-4747
(203) 973-1000
fax: (203) 975-8744
Trinzic Corp.
Waltham, MA
(800) 234-7724
(617) 891-6500
fax: (617) 622-1544
PROGRAMMING TOOLS
PROBLEM CHOICES TRADE-OFFS
Standard programming Use Notes @functions or Short development time
tools don't always @commands; rely on API and low costs yield
do the job. calls for more sophisticated modest, generic
tasks. applications with
limited capabilities.
Write a custom C++ Tight development
program with a c
ontrol and more robust
proprietary mail applications but
interface, middleware increased complexity
layer, and data- and development time.
synchronization code.
MULTIPLE DATA SOURCES
PROBLEM CHOICES TRADE-OFFS
Importing and Use bundled Notes tools. Simple, but limited to
exporting data structured or tabular text
from various from spread sheets and
sources. small databases.
Write a custom C Good for solving specific
program that calls problems versus code that
the Notes API to handle can be reused in other
data import/export. situatio
ns.
Use a third-party Easy point-and-click
product. mapping for source and
target data versus costs
that can run as high
as $25,000 per server,
depending on the product.
MAIL INTEGRATION
PROBLEM CHOICES TRADE-OFFS
Connecting multiple Convert messages Simple to implement but
disconnected to text and import effective for only a
E-mail systems. them into a common handful of different
data store. mail systems.
Use multiple gateways Difficult to administer,
for mail-message but a
ble to handle
conversion. multiple mail systems
at a relatively low cost.
Kelly Trammell is a partner with KPMG Peat Marwick's Strategic Services (Houston, TX), which focuses on workgroup computing and sales-force automation. You can reach him on the Internet or BIX at
editors@bix.com
.
Under Construction
