The advantage here is that once a reseller solves the hard problems in the first implementation, it can repeatedly sell the solution to other companies in that industry segment.
The large consulting arms of data-warehouse vendors sell everything from piecemeal development services to turnkey, packaged data marts. Take all three of these segments into account, and you'll find a market that will reach $6.9 billion by 1999, according to the Gartner Group.
Data-warehouse projects are sparkling reseller opportunities because they require many specialized skills to design, develop, and deploy. While shrink-wrapped warehouse solutions are beginning to reach the market, most warehouse efforts are still custom jobs calling for the integration of a wide range of hardware, software, and network components. The basic pieces include a data model, a warehouse server and applications servers, the middleware layer, a coh
esive network infrastructure, client-side analysis tools, data-scrubbing utilities, data-transport utilities, replication engines, and metadata repositories. But don't let this laundry list intimidate you -- the first five items are the key components.
Modeling Data
Data warehouses separate day-to-day data that production applications use from the historical data that strategic planners use to uncover new sales patterns or spot other trends that can boost business. Another difference: Data-entry professionals
and applications update production data continuously, while data-warehouse applications refresh historical data at set times, usually during off-hours when network and CPU use are light.
One of your first steps will be to decide how to store the data. Any relational DBMS (RDBMS), such as Oracle7, Sybase System 10, or IBM's DB2/400, can work as a warehouse repository. However, some organizations opt for multidimensional DBMSes (MDBMSes) designed for data warehousing,
such as Oracle Express and Red Brick Systems' Red Brick VPT. A traditional RDBMS lets end users view data in two dimensions (e.g., by product and by region). With an MDBMS, you can look at data in multiple dimensions (e.g., by product and by region over time).
On another front, universal servers, from Oracle, Sybase, Informix, Computer Associates, and IBM (see "RDBMSes Get a Make-Over" and "How to Improve RDBMSes," April BYTE), are object-relational hybrids that store not only text but complex objects such as images, animation, and sound. These RDBMSes are important for companies that post multiple data types on Web sites.
"The data warehouses emerging today can potentially support lots of data types in addition to text," says Mike Thompson, MIS manager at Integrated Device Technology (IDT) in Santa Clara, California. "For example, we often have business-critical information arriving in fax format. We could scan these images and store them in a database with some kind of logic behind them."
Since 1996, Thompson and his colleagues have been immersed in a project to construct a data warehouse for several areas of the company, including marketing, sales, order processing, and finance. After buying a ready-made data model from Applied Data Resource Management (see the text box "Packaged Data Model: A Gift for Warehouse Developers" below), the team went to work extracting data from production databases on a variety of host computers. They loaded the data into an Informix data warehouse that ran using a bank of SparcServers from Sun Microsystems.
Next, IDT will deploy the warehouse applications on the company's intranet. IDT chose the Informix product for its standard universal-server capability to handle complex data and because it lets developers embed SQL statements in HTML documents, so Web users can see query results dynamically displayed. Donald DePalma, an analyst at Forrester Research, says such multimedia warehouses will soon be the norm.
What's in the Middle
Mi
ddleware connects warehouse databases and front-end decision-support tools. While standard database middleware can handle this task, specialized middleware for data warehouses is arriving. Colin White, principal consultant at DataBase Associates International, a database and data-warehouse consultancy, says specialized warehouse middleware can help companies monitor, track, and control access to warehouse data. "Users need to access data belonging to other departments for cross-business function analysis," White explains.
Some middleware products, including Sybase IQ and Information Builders' popular EDA, offer copy management (also called data staging) to select, edit, summarize, combine, and load the data warehouse with information from operational databases. Quality-analysis programs and filters identify patterns and data structures in the operational data. The patterns help summarize the data and construct views useful for analysis and reporting.
Network Considerations
Data
warehouses typically imply a distributed-data architecture, with bulk transfers of data during off-hours and heavy interactive querying at peak hours of the day. Without proper planning, the performance of the network can suffer. Here's where resellers can help.
There are two primary methods for populating the warehouse with data: bulk downloads, in which the entire database is refreshed on a periodic basis, and change-based replication, where the system copies over just the changes.
Transmitting only the changes puts less stress on the network but requires more complex programming to set up. Bulk downloads can heavily tax the network -- a 20-GB database implies a 20-GB transfer -- but such warehouses are easier to set up and maintain.
In either case, warehouse architects must pay close attention to the frequency and scheduling of data updates (perhaps job-scheduling software is required). It's usually easy to determine the optimum time for refreshing the warehouse with new data, such as fol
lowing a billing or sales cycle.
Browser Access
The final piece of the data-warehouse package is end-user access. Increasingly today, that means a Web connection such as the one Rand built when it linked a data warehouse and an intranet-based reporting system called Oasis. Rand, a nonprofit research firm, helps organizations develop public-policy strategies.
"Instead of having a copy of warehouse-reporting applications on every desktop, users can access server-based versions through their Web browsers," says Ken Krug, a treasurer at Rand. "This streamlines electronic access to corporate financial and man-power data."
Rand created Oasis on top of the Oracle7 database and Oracle Project Accounting software. Andersen Consulting helped with the initial financial-software selection. Oasis incorporates statistical and project-accounting programs built using the Oracle software in conjunction with Microsoft Excel and Netscape Web Server. Every two weeks, data from Oracle Project
Accounting flows to an Oracle7 data warehouse that Rand uses to generate dozens of ad hoc reports.
Phase 1 of Oasis includes 45 reports accessible via the company's intranet to help users track the thousands of simultaneous projects and tasks ongoing at the organization. These reports are available anytime at all via Netscape Navigator.
A new generation of decision-support tools is helping VARs build next-generation applications, such as MicroStrategy's DSS Web, an analytical engine for translating user queries from Web browsers into an optimal SQL execution plan. Other products in this category include Oracle Express Server, Information Builders' WebFocus, Seagate Software's Crystal Reports, BusinessObjects' soon-to-be-released Darwin, and IQ Software's IQ LiveWeb.
Expanding Opportunities
The Internet's flexibility as an information-delivery vehicle and the growing cultivation of corporate data will continue to mean much custom development work for resellers. Even in an e
ra of shrink-wrapped solutions, each data warehouse requires a unique architecture and includes a specific set of business requirements.
"For us, it comes back to what's best for our customers," says
Rick Roy
, vice president/information products division of reseller M&I Data Services. "Our real thrust with each data-warehouse initiative is to help companies identify the information that is truly important to them."
Where to Find
Business Objects
Cupertino, CA
Phone: 408-973-9300
Internet:
http://www.businessobjects.com
Planning and Implementing a Data Warehouse
