Indexing Gets Smart

Dennis Allen

Information publishers are turning to a new breed of smart-indexing tools that can automatically summarize and condense huge documents without human intervention. Automatic-abstracting technologies address problems that even complex Boolean searches can't.

On-line publishers of information are evaluating smart-indexing tools from companies such as Iconovex (Bloomington, MN, (612) 943-0292) and InText Systems (Folsom, CA, (415) 391-5290) so that content subscribers won't waste time needlessly downloading large documents. "A lot of times, full-text indexes provide a lot more information than people can use," says Tom Mandt, director of advanced development at IVI Publishing (Minneapolis, MN), a digital publisher that's expanding its on-line presence. "The ability of these tools to look at a lot of information a nd condense it is intriguing," he adds.

Iconovex, which already sells a series of programs, called Indexicon ($149), that automatically generate back-of-the-book-style indexes for popular Windows and Mac word processors, has developed a toolkit for C and C++ programmers, called Syntactica. Available now for Windows, and soon for Unix and the Mac, Syntactica can generate an indicative abstract (i.e., a summary of the full text) by using a combination of semantic and syntactic analysis. In addition to creating traditional back-of-the-book indexes, the toolkit can create concept indexes, in which a document's key ideas are provided both alphabetically and by page number.

Another intriguing development comes from InText Systems. The company is currently developing a software development kit for on-line publishers to use in writing agent programs that let users launch searches through World Wide Web documents. The agents create abstracts of documents that meet a specific search criterion.

Anot her company using smart-indexing technologies to summarize text is Oracle (Redwood Shores, CA, (415) 506-7000), which says it will integrate its ConText linguistic-analysis and content-extraction software into its Documents workgroup software for managing unstructured text data.

"The whole idea of the intelligent document is gaining acceptance," says Carl Frappaolo, executive vice president at Delphi Consulting (Boston, MA), a consultancy specializing in electronic-document management. "Smart-indexing tools are the next big thing in this area."