As processors become more powerful, networks more ubiquitous, and data types more rich and varied, storage technologies struggle to keep pace
Scott Wallace
Your computing environment contains many types of storage, from the registers that feed the processor pipelines to the jukeboxes that store archival data. The different types of storage in a computer system form a hierarchy, with fast, expensive devices at the top and slow, inexpensive ones at the bottom.
The best visual representation of this storage hierarchy is a pyramid. Since the early days of mainframes, computer systems engineers, managers, and users have found the pyramid ideal for depicting how data is organized and distributed across storage devices. Over the years, the simple RAM-disk-tape pyramid has evolved into a complex, dist
ributed, hierarchical data-storage architecture populated by devices of varying capacity, performance, reliability, and price.
Today, the volume of data managed by this storage hierarchy is rising dramatically. The ever-increasing storage demands of individual applications; the migration of large database applications from the glass house onto corporate networks; and the introduction of new groupware, imaging, and multimedia desktop applications contribute to the need for more distributed-storage capacity and more effective tools to manage data. The goal is cost-effective data management that properly safeguards information while providing users with appropriate access at the lowest cost.
Fortunately, rises in storage requirements have been paced by rises in storage capacity. Recently, hard drives have shown a 60 percent annual increase in data density, a trend that shows no sign of changing (see the figure "Single-Platter Capacity"). MO (magneto-optical) drives doubled their capacity not long a
go and will do so again soon; tape subsystems in several form factors have dramatically increased storage capacities and speed; and jukeboxes, or auto-loaders, have improved in price, performance, and reliability, further leveraging MO and tape drive advances. The result is a storage hierarchy that is becoming at once more flexible, more responsive, and more complex.
Storage Hierarchy
Every organization must create and configure its storage subsystems based on user needs and compatibility with the existing IT (information technology) infrastructure. Each organization must, in effect, design and construct its own storage pyramid. Typically, this is done by selecting products with a range of per-megabyte costs and access times that provide a price/performance continuum by mixing slower, low-cost products with faster, high-cost products (see the figure "Cost vs. Speed").
Although many storage products today have short development and life cycles, this hasn't prevented enterprises from effectivel
y incorporating technological developments into their storage hierarchies. "For most organizations, the 'sweet spots' don't move around very much, because all components of the pyramid are improving performance more or less simultaneously," says Alex Nedzel, a consultant with Ernst & Young's Center for Information Technology and Strategy in Boston, Massachusetts.
A bigger problem is the proliferation of data, where there are two basic concerns. "First, there's the relatively straightforward problem of getting enough data on-line at the right cost," Nedzel says. "But the more complicated issue is how to assure the safety of that data from both a confidentiality and a disaster-recovery standpoint." While industrial-strength software to manage storage exists in mainframe environments, none exists that can manage storage in the type of heterogeneous distributed-computing environment that typifies many client/server systems.
To match such an environment, contemporary storage hierarchies are also broa
d and heterogeneous. To confuse matters more, they often differ from platform to platform, even in a single organization. A mainframe storage hierarchy is different than a desktop hierarchy, which is different than a server, laptop, or PDA (personal digital assistant) hierarchy. A pyramid combining all platforms would have memory, magnetic disk, MO disk, floppy disk, tape, and auto-loader layers (see the figure "Storage Hierarchy").
The Persistence of Memory
The top of the storage pyramid consists mostly of DRAM and associated caches that are used for temporary storage of programs and data. As applications and operating systems become more memory hungry, you'll see the average memory of desktop systems rise to accommodate such requirements.
The real news is flash memory. Flash memory has been around for some time but is just now becoming popular. This is a result of many factors, primarily increased chip-storage capacity, reduced manufacturing costs, and demands for low-power, low-weight stor
age created by PDAs and other mobile-computing devices.
Flash memory is not only fast (with a less-than-1-millisecond seek time), it's extremely reliable. Because it has a lower soft-error rate than DRAM and is nonvolatile, flash memory is ideal for BIOS and critical systems applications. Flash memory today has low power consumption (e.g., Intel's 16-Mb FlashFile chip draws only 1 milliamp in static mode or 1 microamp in power-down mode). And because a flash-memory system has no moving parts, it offers survivability well beyond that of rotating media--it can survive a shock in excess of 1200 g's and operate in temperatures ranging from -13û F to +167û F. Flash storage is also small (32-Mb chips should soon provide storage of 195 MB per cubic inch) and light, and it operates silently. Flash memory today is an ideal storage medium for laptops, as well as personal and mobile digital applications.
The problem with flash memory is price. An OEM can expect to pay $1200 for 40 MB of flash memory. Thus,
typically, flash-memory disk purchasers buy 5- or 10-MB PCMCIA cards (at OEM pricing of $200 to $300) for portable computing environments. The relatively low storage requirements of PDA devices fit flash-memory pricing. In addition, PDA operating systems are designed from the ground up to support flash architectures. "Flash is block-erasable but byte-writable. It's not like a disk drive that has direct overwrite," explains Bruce Bonner, flash drive product-line manager for Intel in Santa Clara, California. "With flash, you want programs and data separated. Right now, desktop operating systems commingle them; they're not flash-friendly."
Desktop operating systems will exhibit flash-friendliness in the next year or two. And if what Nelson Chan, director of marketing at SunDisk in Santa Clara, California, says is correct, almost every desktop machine in the future will have a PCMCIA slot for transferring data between the desktop and portable computing devices. "We expect to bring a low-cost PCMCIA deskto
p card reader to market soon with a street price of under $80," says Chan. With sustained read rates of 3 or 4 MBps (writing is slower, on the order of 200 KBps), future PCMCIA flash-memory disks--like floppy disks today--may be a key portable-to-desktop end-user link.
Spin Doctors
Hard disk technology is the most rapidly evolving storage technology, largely as a result of the size of the hard disk market, its highly competitive character, and industrywide R&D spending on rigid disks. Phil Devin, vice president of storage technologies at Dataquest in San Jose, California, foresees density improvements of 60 percent to 65 percent annually, prices falling by 12 percent per quarter, and product life cycles of less than a year. A key factor in density improvements will come from developments in MR (magnetoresistive) heads, with much of the work being done by IBM.
MR heads allow greater areal density than either thin-film or ferrite-inductive heads (see the figure "Areal Densities"). According to
Bob Scranton, director of storage systems and technology at IBM's Almaden Research Center and director of advanced technology at IBM's San Jose, California, Storage Systems Division, MR heads will be important for maintaining a 60 percent compound annual growth rate in areal density for the foreseeable future. "With expected improvements in our MR heads, the focus will shift more toward the electronics," says Scranton. IBM is just one of many companies pursuing research into the data-recording, digital-read-channel, and interface technologies that will help keep areal-density growth at historic levels (see "Digital Hard Drives" on page 91).
Data density isn't the only good news: Drive latency, a function of rotation speed, is getting better, too. "We've seen an increase in spin speed from 5400 rpm to 7200 rpm," says Paul Wasenberg, product-line marketing manager for Micropolis of Chatsworth, California. Spindles spinning 33 percent faster means data is available sooner. Combined with increased areal de
nsity (which also means an increase in linear density), the effect is a higher data rate. The trade-off is noisier, hotter, and more power-hungry drives.
IBM's DFMS and DFHS families of high-performance 3 1/2-inch drives rotate at 7200 rpm. These 1-, 2.1-, and 4.3-GB drives feature an industry-leading areal density of 564 Mb per square inch, a seek time of 8.6 ms, and the industry's highest media rate of 12.2 MBps. The Barracuda, another 7200-rpm drive from Seagate Technology (Scotts Valley, CA), offers 8-ms seek times and a capacity of 4.1 GB. With lower seek times, improvements in caching, and higher spin speeds, access times will only get better.
Focus on Optics
The principal advantages optical disks (rewritable and WORM) have over hard disks are removability and greater bit density. These combine to make optical storage ideal for library and archive applications. In the past, the optical-storage market was primarily focused on large 12- and 14-inch drives, but the emphasis today is on sma
ller products (see "Optical Advances" on page 107). Projections for 5 1/4-inch and 3 1/2-inch rewritable optical drives indicate strong market growth (see the figure "Magneto-Optical Drive Sales").
The 5 1/4-inch form factor is attractive for desktop applications. "In 1993, we saw the introduction of the double-capacity 5 1/4-inch drives, which significantly improved the cost per megabyte of optical storage over the earlier 650-MB models," says Stan Corker, director of removable storage research at IDC (San Diego, CA). And things are likely to get better. Hitachi, outpacing optical-drive manufacturers Hewlett-Packard, Sony, and IBM, has released its triple-density 2-GB 5 1/4-inch drive.
Rewritable optical drives in the 3 1/2-inch "rigid floppy" form factor are also becoming more capacious and attractive. "The momentum is with
the 3 1/2-inch drive," says Patty Chan, optical-storage analyst with Dataquest. "It's a smaller form factor, the entry cost is much lower, and it can be used in many m
ore applications. We expect 1993 figures to show 3 1/2-inch drives surpassing 5 1/4-inch drives in unit shipments, and in 1995 we anticipate some 3 1/2-inch drives in excess of 500 MB."
Because optical is a removable storage medium, there is a strong demand for interchangeability standards. "The development of standards probably tends to throttle back the rate of technological improvement," comments IBM's Scranton, referring to the disparity in bit-density improvements between magnetic and optical storage over the past several years. "CD-ROM players, for instance, have been at the same density for many years. The reason is media interchange standards."
In terms of unit shipments, however, CD-ROM is the fastest-growing segment of the optical marketplace (see the figure "CD-ROM Breaks Through"). Until recently, CD drives were strictly aftermarket products, typically oriented toward publishing and data-distribution applications rather than storage. "CD-ROM has graduated from an aftermarket item to
an option in the standard configuration," notes Dataquest's Chan.
End-user and OEM interest in CDs is being driven primarily by pricing, but functionality helps. "Over the next two years, the price of CD-R [CD Recordable]--now around $5000--will drop fast. In that same time frame, we can expect to see erasable CDs," says Chan. These developments will likely push CD-ROM drive prices lower.
Tale of the Tape
Magnetic tape products are growing quickly in capacity and, despite impressive improvements in hard disk storage, are keeping pace with magnetic drives. A critical end-user issue with tape is storing data faster, particularly in server environments where the window for backing up data is constrained, but the amount of data to be backed up keeps growing.
QIC (quarter-inch cartridge) drives account for 76 percent of the installed base of all computer tape drives, according to Robert Abraham, an analyst with Freeman Associates in Santa Barbara, California. "This momentum--coupled with a
strong desire for backward compatibility, well-defined performance migration paths, and low costs--has discouraged many users from defecting to helical-scan or optical technologies," he says.
QIC tape products can be configured to compatibly serve a range of computing environments, from mainframes to laptops. "The highest growth area is the
QIC 3 1/2-inch format, and that's being driven by PC users," says Fara Yale, tape market analyst at Dataquest. Yale estimates 1993 shipments of
QIC 3 1/2-inch floppy interface products at more than 2 million units, mostly for stand-alone PC backup. Desktop penetration, however, remains low, at less than 10 percent of desktops.
Arguably the fastest-growing tape market segment is parallel-port interface tape systems; these products can support not only desktop systems but laptops as well. A variety of manufacturers, including Colorado Memory Systems, Conner Peripherals, and Tandberg Data, offer parallel-port interface products.
DAT (digital aud
iotape)--because of its storage density and small form factor--is popular for data-intensive storage applications at the desktop, the workstation, and the server level. Products by Conner Peripherals, HP, Sony, and others are making inroads into the tape-storage market.
At the high-end, DEC's DLT (Digital Linear Tape) offers capacity, speed, and extreme reliability. The design allows for a head life of 10,000 hours (nearly five times that of previous drives), a recommended average of 10,000 reads/writes per cartridge, and an MTBF (mean time between failures) of 80,000 hours.
The Storage Boss
Managing the storage hierarchy--whether for a PDA, laptop, workstation, server, or mainframe--is getting more complex, and the consequences of mismanagement are getting more expensive. A survey of 450 information systems executives at Fortune 1000 companies found computer downtime cost an average of $78,191 per hour and occurred, on average, nine times a year. A typical "outage" cost $330,000, including t
he costs of recovering or reconstructing data. Clearly, users must prepare themselves and their data for the inevitable.
In the client/server environment, this preparation can be complicated, and hierarchical storage management systems are being called on to automate backup and recovery, manage file migration, and oversee volume and library management services. The goal is to create a high-reliability, high-access, high-performance data management environment. "Hierarchical storage management comes from the mainframe world, and users migrating to client/server are looking for the same kind of management tools they had on their mainframes," says Barbara Goldworm, product-line manager for management services at Novell in Provo, Utah.
NDMS (NetWare Distributed Management Services), Novell's management strategy, supports the utilities and functions to allow decentralized management of storage and data across distributed environments. Within NDMS today, HCSS (High-Capacity Storage System Services) su
pports hierarchical management across the magnetic disk and optical disk layers. "The Novell strategy is to provide key management services within NDMS, plus APIs to development partners for extended or specialized additional services," Goldworm notes. HCSS lets users set "watermarks" on disk capacity and trigger transparent data migration to and from magnetic and optical storage.
Conner Software Products (Lake Mary, FL) offers hierarchical storage management software called Conner HSM. The Conner system is designed specifically for automated data management on Novell NetWare networks. It manages three of the storage pyramid layers: magnetic disk, optical disk, and tape. Conner HSM, which is typically configured with a dedicated storage server running as its primary task, has three logical components: the network interface, migration and data management software, and analysis and reporting to support assessment of migration thresholds. "From the user's perspective and from the network administrator's p
erspective, Conner HSM is just one large storage pool," explains Rick Luttrall, product marketing manager for Conner's advanced network products. "They really don't know where the files reside within the hierarchy; that's transparent."
One of the problems confronting prospective storage management customers is an uncertainty about their true storage needs. With individual desktop applications expanding data requirements, mainframe applications migrating onto networks and desktops, and altogether new applications emerging, it's difficult to project storage needs. "We provide a tool, HSM Planner, that helps users analyze operations across the enterprise--all the servers on the network--and determine what their storage requirements are and are likely to be within the next three years," says Luttrall.
Storage Directions
What can we expect in the future? Flash memory will become less expensive and smaller and increase in capacity. As desktop and laptop operating systems become "flash friendly," lo
ok for operating systems on a chip. Also, solid-state disks, such as DEC's new 580-MB DRAM ESP580, can be expected to penetrate specialty markets like high-speed, high-value transaction-processing applications, where data integrity is paramount and I/O storage is the bottleneck for high-value transactions.
In hard drives, you should look for big-drive capacity (i.e., 500 MB) on small drives and lower per-megabyte cost everywhere. Also, advances in interface technology will make it easier and less expensive to use these large-capacity drives (see "IDE Takes Off" on page 97). MO storage will experience faster density increases than it has in the past, while in CD-ROM, prices will continue to fall and multispeed drives will proliferate, with triple- and quadruple-speed drives from companies such as NEC and Pioneer becoming increasingly common.
On the cutting edge, look for holographic storage products to come to market within the next year. Tamarack Storage Devices of Austin, Texas, a spin-off of M
CC, expects to ship its holographic product, called MultiStore, in the second quarter of this year. MultiStore will provide 30 GB of removable WORM storage at an initial end-user cost of $6000, dropping to $3500 with volume. Media cost is projected to be $5 per gigabyte. The device is essentially an auto-loader for 30 2 1/2-inch disks, each of which stores a gigabyte of data on a Du Pont photopolymer material similar to that used in holographic art.
On the software side, hierarchical storage management software will get more robust, address more layers of the pyramid, and better manage storage devices. In the next three years, you'll see capable, multiplatform products that offer mainframe-quality data-storage management in distributed environments.
All these improvements will be necessary, as the future will bring more and more data that's increasingly critical to success in a wide variety of businesses and industries. "Keeping your detailed manufacturing data on-line so you can optimize produc
tion efficiency and products, for instance, is a competitive investment," notes IBM's Scranton. "If it produces improved quality and reduced cycle time, then more companies will be using data as a competitive weapon." The result is an accelerating demand for products to ease the management of that data.
Figure: Single-Platter Capacity
For the foreseeable future, hard drive capacities will continue to advance an average of 60 percent per year, regardless of form factor. (Source: Dataquest, 1993)
Figure: Cost vs. Speed
With nonvolatile memory, access time and cost are inversely proportional. You can pay up to $40 per megabyte for flash storage, while most forms of tape storage are well under $1 per megabyte. (Source: Freeman Associates, Inc., 1993)
Figure: Storage Hierarchy
A pyramid is ideal for representing the relationships between different parts of the storage hierarchy. Technologies at the top are more expensive and are fielded in small capacities; tec
hnologies at the bottom are less expensive and have larger capacities.
Figure: Areal Densities
Head technology is one of the driving forces behind hard drive capacity increases. Magnetoresistive heads are expected to accelerate capacity increases, partly because they work so well with newer digital read technologies. (Source: IBM Corp., 1992)
Figure: Magneto-Optical Drive Sales
As capacities increase, so will the popularity of MO technology. The plateau in 5 1/4-inch shipments from 1990 to 1992 reflects the effects of the standards turmoil in that market. (Source: Disk/Trend, Inc., 1993)
Figure: CD-ROM Breaks Through
After years of slow growth, CD-ROM players are starting to be considered standard equipment on desktop PCs. (Source: Disk/Trend, Inc., 1993)
Scott Wallace is a BYTE technical editor. He can be reached on BIX c/o "editors."
Managing Mass Storage
