Archives
 
 
 
  Special
 
 
 
  About Us
 
 
 

Newsletter
Free E-mail Newsletter from BYTE.com

 
    
           
Visit the home page Browse the four-year online archive Download platform-neutral CPU/FPU benchmarks Find information for advertisers, authors, vendors, subscribers Request free information on products written about or advertised in BYTE Submit a press release, or scan recent announcements Talk with BYTE's staff and readers about products and technologies

ArticlesLessons Learned in the Data Mine


February 1997 / Bits / Lessons Learned in the Data Mine

Alan Paller , director of research at the Data Warehousing Institute, tells how to avoid the pits in your data-spelunking adventure.

Dave Andrews

BYTE: What are the most common problems and solutions in a successful data warehouse implementation?

Paller: One of the more difficult problems, especially in a large data warehouse, is getting the various divisions of a corporation to arrive at a common set of definitions for data, for objects likes sales, customers, and products. Take an insurance company, for example. It has a property and casualty division, it probably has a health and life division. Those divisions have completely different kinds of products. So the data you keep for the products is completely different from one division to the other. Defining what exactly is the "customer" becomes difficult. Is the customer a company or is the customer an individual? Or is the customer that piece of building you are insuring?

There are good reasons for the differences in definitions, and there's no easy solution to how you create a corporate-wide common set of data definitions. The answer to the question of how you deal with that problem is not a fun one but it's the simplest one anyone has ever found: you agree to disagree, and you build separate data marts for each of the divisions. Then you find the few things that are common, like payables or receivables -- organizations that you owe money to or collect money from -- and you make corporate-wide databases with those. You get an added bonus when you identify common suppliers: you can draw all of your purchasing together to get one of the most important and profitable benefits of a data warehouse, wh ich is being able to show your vendors all you're buying from them, which lets you get better discount rates.

BYTE: What's another common 'gotcha', and how can you overcome it?

Paller: Another one is one of my favorites, which is when a pair or group of vendors say, "Our products will work well together." It's quite common to hear from a vendor that it has a strategic alliance with another vendor and that the products will work together. Well, many times, they don't. So, the only time you get a strategic alliance that is viable is when it's client-driven. When a customer says "I will buy these two products if and only if they work well together and if and only if you agree to make the interface between the products part of your product." But the customer has to be mighty big to persuade a company to do this. You should also make the vendors prove that their products work together, and guarantee in writing that they continue to work well together for at least three years.

BYTE: What are other caveats for customers who've already rolled out their data warehouses?

Paller: The reality is as soon as they are worthwhile, people, very important people, become dependent on them. And so all of the things that you have to do for a big business system become important. You have to have security that's real, you need scheduling to automatically update it and verify the update, and you have to have backup and recovery. You have to have centralized management for distributed servers. The amount of money that people are spending on that part of data warehousing will grow probably to 50 percent of all investment of data warehousing. Most people don't budget for this, but they should.


For more information on the Data Warehousing Institute, send e-mail to tdwi@aol.com .


Alan Paller, Data Warehous ing Institute

photo_link (154 Kbytes)


Up to the Bits section contentsGo to previous article: SearchSend a comment on this articleSubscribe to BYTE or BYTE on CD-ROM  
Flexible C++
Matthew Wilson
My approach to software engineering is far more pragmatic than it is theoretical--and no language better exemplifies this than C++.

more...

BYTE Digest

BYTE Digest editors every month analyze and evaluate the best articles from Information Week, EE Times, Dr. Dobb's Journal, Network Computing, Sys Admin, and dozens of other CMP publications—bringing you critical news and information about wireless communication, computer security, software development, embedded systems, and more!

Find out more

BYTE.com Store

BYTE CD-ROM
NOW, on one CD-ROM, you can instantly access more than 8 years of BYTE.
 
The Best of BYTE Volume 1: Programming Languages
The Best of BYTE
Volume 1: Programming Languages
In this issue of Best of BYTE, we bring together some of the leading programming language designers and implementors...

Copyright © 2005 CMP Media LLC, Privacy Policy, Your California Privacy rights, Terms of Service
Site comments: webmaster@byte.com
SDMG Web Sites: BYTE.com, C/C++ Users Journal, Dr. Dobb's Journal, MSDN Magazine, New Architect, SD Expo, SD Magazine, Sys Admin, The Perl Journal, UnixReview.com, Windows Developer Network