High-performance servers will start using a new I/O architecture to boost performance without costing a fortune.
Tom Thompson
Today's PC servers face ever-daunting demands on their resources. Beside traditional roles of file, mail, and print services, such machines must now handle new tasks, such as database queries, on-line transaction processing, and streaming video for multimedia applications. The explosive growth of the Internet and corporate intranets hasn't helped: Now servers must manage numerous high-speed network connections and churn out graphics and Ja
va applets for content-rich Web applications. A PC server's architecture wasn't designed to deal with the large throughput that these tasks demand. Boosting throughput to handle these loads has meant adding more hardware to the server, such as more processors for symmetric multiprocessing (SMP) and specialized (read: expensive) high-speed peripherals. How far you could improve your corporate network's capacity going this route has been determined by the size of your equipment budget.
However, a recently introduced I/O architecture called Intelligent I/O, or I
2
O, changes the situation. I
2
O-compliant servers will be able to administer more tasks despite a limited amount of hardware because the architecture off-loads portions of the work onto intelligent I/O subsystems. Dedicated I/O processors (IOPs) on these subsystems take care of the grit
ty details of interrupt handling, buffering, and data transfers. This improves the server's I/O throughput and frees the server'
s main processors so that they can handle more critical tasks.
I
2
O in a Nutshell
An independent standards body known as the I
2
O Special Interest Group (SIG) manages I
2
O's
architecture
and specifications; see the group's Web site at
http://www.I2Osig.org/
. The standard has widespread support, with the list of members on the steering committee reading like a Who's Who of the computer industry. System OEMs such as Compaq, NetFrame, and Hewlett-Packard are on it, as are OS vendors such as Microsoft, Novell, and the Santa Cruz Operation (SCO). Networking companies such as Bay Networks, Cabletron Systems, and Eicon Technology are also members. Version 1.5 of the I
2
O specification was approved by SIG members this March.
While we have covered I
2
O before (see "Smarter and Faster I/O for Servers" in the May BYTE), a brief summary won't hurt. I
2
O features a hardware-independent architecture centered around a "split driver" model. An I
2
O driver consists of an OS-specific module (OSM) and a hardware device module (HDM). The OSM manages OS-specific details such as the file system or higher-level network protocols, while the HDM understands device arcana, such as control register addresses and I/O port addresses, and deals with interrupt handling. The OSM typically operates in main memory as an OS process, while the HDM executes on an IOP. A server's firmware must be modified so that when it scans the PCI buses for devices at boot time, it recognizes I
2
O-compliant devices and uses a different procedure to install their drivers, as shown in the figure
"Boot Sequence Using I
2
O"
.
The modules communicate by passing messages (typically pointers to data) t
hrough a communications layer, which is actually a queue. There is a standard set of message types for block storage devices (hard disks and CD-ROM drives), network interfaces (Ethernet and Fiber Distributed Data Interface, or FDDI), RAID arrays, and other services. OSMs will be implemented in Windows NT 5.0 as a DLL and as a NetWare loadable module (NLM) in Novell's IntranetWare. The HDM portion of the I
2
O architecture is currently built around Intel's i960 RP and i960 RD IOPs. These IOPs run Wind River's IxWorks, a multithreaded real-time OS (RTOS). IxWorks implements the object-oriented API described by the I
2
O standard, which simplifies driver design. Because the driver halves converse by passing messages, there is no reason other IOPs could not be used. Furthermore, the common communications interface allows, say, a block-storage OSM to interoperate with the HDM of any vendor's mass storage device.
The Throughput Problem
Because of I
2
O's divide-and-conquer
strategy, it promises to wring more performance out of existing hardware. To understand why, we have to look at two big issues that server architectures must face: availability and scalability. Availability is where the server has sufficient resources (memory, disk drives, network interfaces, and processor capacity) to instantly process requests. Given enough users bombarding a server with requests, such resources become scarce, and the server's availability then plummets.
Scalability means that when you do add more resources to a server, the cumulative effect on its performance should be linear. This approach works--up to a certain point. Adding more resources to the server means that more devices compete for the same system bus. It can become so congested that the additional processors and peripherals wind up waiting for a chance on the bus. Now the system bus is the scarce resource, and adding extra hardware does little to improve performance. One solution is to add more buses to the system--a techni
que supercomputer designers have used (see "The World's Fastest Computers," January 1996 BYTE). This remedy works, but the extra hardware and the complex design drive up the system's price tag.
I/O exerts an effect on a design's scalability in another way. The server's main processor--and thereby its buses--can be tied up for thousands of cycles when performing a low-level I/O task such as executing a network interrupt handler. Furthermore, the jumps from the server application to the interrupt handler and back can create cache misses. For 200-MHz or faster processors, the pipeline delays that occur while the cache refills from slower main memory add up to more lost cycles, thus hampering performance.
I
2
O enables the efficient use of the server bus because the IOP fields the low-level interrupts in local memory provided for it. Furthermore, the HDM uses the bus only when transferring data in bulk to or from main memory. With a less congested bus, the server hardware can handle more task
s. The performance improvements can be dramatic: "Some of our tests have shown an I
2
O implementation confers a 495 percent efficiency gain for Windows NT throughput," says David Miller of Xpoint Technologies, a company working on I
2
O products. This additional capacity also contributes to better scalability for the server. "While a conventional dual-Pentium Pro server can manage only two Fast Ethernet cards, both at 40 Mbps, with an I
2
O-compliant system and our software you can have several Fast Ethernet cards and operate them at full capacity [nearly 100 Mbps]," Miller adds. I
2
O also specifies a unique mode of operation, termed peer-to-peer, where devices can communicate directly with one another with little intervention from the server OS. (See the sidebar "By Your Peers" for more details.)
As an enabling technology, I
2
O's capabilities can be used to provide a host of services, both new and old, with little impact on the server's throughput. Certai
n of these services would be implemented either as intermediate service modules (ISMs) or through peer-to-peer services. For network operations, I
2
O can off-load repetitive chores that make heavy use of interrupts onto smart network interface cards (NICs). At a minimum, the NIC's IOP would execute an ISM that implements the algorithms used to encrypt or decrypt secure data streams, sparing the server's host processors from this computational overhead. However, given a fast IOP, it's possible to go a lot further in relieving the server of network operations. For example, another ISM would handle the handshake used in a firewall's verification process, and any associated IP security. Other ISMs could manage HTTP lookups and process FTP transfers. Finally, in concert with peer-to-peer transfers with a hard drive, the IOP could facilitate Web page caching. Thus, through I
2
O ISMs and peer-to-peer services, most of the I/O-intensive operations of both a Web server and a proxy server could be c
ombined on the same NIC. Peer-to-peer operations could be used to improve network reliability by implementing load balancing on multiport NICs. If a LAN segment fails on a network, an ISM on the NIC could detect this, examine the incoming network packets, and reroute them to a port that's connected to the backup LAN segment.
For disk storage, an IOP could execute an ISM that implements RAID functions. The ISM would then operate I
2
O-compliant hard drives as parts of a RAID array. While such a design lacks the speed of a dedicated RAID controller board, it allows low-cost servers to reap the benefits of RAID's storage integrity. Other ISMs could perform on-the-fly data compression/decompression between the OS and the hard drive.
Devil in the Details
The I
2
O specification is broad, so a wide range of system designs is possible, each addressing a different market (see
"Types of I
2
O Designs"
). Supermicro's Super P6DNH and Microconics Compu
ters' M6DPd incorporate an i960 IOP on the main logic board for a low-cost I
2
O implementation. The on-board IOP performs most of the interrupt handling for "dumb" peripherals, and so improves throughput. The trade-off is that while the design is cost-effective, it's not very scalable. Another issue with this design is how much intelligence for I/O handling should be moved to the IOP. Says Gary Abbott, server technology strategist at Dell Computer: "Compared to the clock speeds of IOPs, chip designers are rapidly boosting the speed of the main processors. In six months, the increase in speed of the main processor could negate any advantage to executing code on an IOP." Other system designers agree. "Compaq focuses on investment protection for our server customers; therefore, the fixed performance of an IOP on a motherboard has long-term problems," comments Karl Walker, director of technology development for server products at Compaq. Although Compaq won't comment on future products, the IOP could b
e placed on a removable daughtercard, which would extend the system's lifetime for the customer. Still, the lone IOP can be of great value processing lightweight, highly repetitive operations that involve lots of interrupts. This reduces the effects of heavy I/O traffic on the server, and thus improves its availability.
Another server design has each peripheral board supplying its own built-in IOP. This enables concurrent I/O operations to occur in the server, thus boosting its availability and scalability. If a busy peripheral happens to be on a secondary PCI bus, the PCI bridge chip can filter out its bus traffic, thus keeping the primary PCI bus clear so that it can manage other high-speed peripherals. The IOP adds about $50 to the cost of the peripheral card, but that's less than the cost of adding another main processor to the system (about $500). For certain peripherals, such as Gigabit Ethernet devices, the inclusion of an IOP is virtually assured, since a dedicated processor is required to manag
e the interface's high throughput.
From the OS vendor's point of view, adding support for I
2
O isn't difficult. Says Michael Rex of Novell, "Because of IntranetWare's modular design, we don't have to modify the OS. We are providing I
2
O OSMs that will run on our existing OS product." To IntranetWare, OSMs are simply NLMs. Specifically, software engineers revised existing disk and LAN drivers so that they convert standard requests into I
2
O messages. The messages are then passed to a specific HDM, which speaks to the hardware. Support NLMs for handling PCI operations and the I
2
O device registry were also added. The peripheral vendor provides the HDM, typically in firmware on the PCI expansion card. Novell is working closely with peripheral vendors to ensure a good fit between OSMs and HDMs. The company is a key supporter of the I
2
O SIG.
The Changing State of Servers
I
2
O improves a PC server's availability and scalability by shi
fting most of the I/O interrupt handling onto a less costly IOP, which makes it attractive to both IS managers and system vendors. Equally attractive is the fact that this performance win can be done with only minor revisions to the server OS and no modifications to existing applications.
Since the 1.5 spec was approved early this year, there are just a few I
2
O-compliant peripherals available at this time. But this situation should change as high-speed network cards using Gigabit Ethernet become more common, and server OEMs modify their boot firmware to recognize I
2
O devices. Peer-to-peer implementations won't arrive until late next year, but when they do, they have the potential to radically change how servers get their work done.
Where to Find
Compaq Computer
Houston, TX
Phone: 800-652-6672
Internet:
http://www.compaq.com
Dell Computer
Round Rock, TX
Phone: 800-388-8542
Internet:
http://www.dell.com
Hewlett-Packard
Palo Alto, CA
Phone: 800-475-6697
Internet:
http://www.hp.com
Intel
Santa Clara, CA
Phone: 800-628-8686
Internet:
http://www.intel.com
Microsoft
Redmond, WA
Phone: 800-426-9400
Ph
one: 206-882-8080
Internet:
http://www.microsoft.com
NetFrame
Milpitas, CA
Phone: 800-737-8377
Internet:
http://www.netframe.com
Novell
Provo, UT
Phone: 888-321-4272
Internet:
http://www.novell.com
Xpoint Technologies
Boca Raton, FL
Phone: 561-241-8447
Internet:
http://www.xpoint-tech.com
Single IOP on main logic board
PROS:
* Low cost
* Improved availability because I/O interrupts handled by I/O processor IOP)
CONS:
* Little to no concurrent processing
* Limited scalability
IOPs on peripheral cards
PROS:
* Concurrent processing possible
* Good scalability--more peripherals add more IOPs
* PCI bridge chip can filter traffic to minimize bus congestion
CONS:
* Raises price of peripheral boards
* Overkill in a small server
illustration_link (26 Kbytes)
The I
2
O architecture is flexible, and it's capable of abstracting many layers of I/O controllers from the OS.
illustration_link (31 Kbytes)
New boot firmware that recognizes I
2
O devices is necessary because they're owned by an IOP and use a different setup sequence.
Tom Thompson is a BYTE sen
ior editor. You can reach him at
tom_thompson@bix.com
.
I
2
O Beats I/O Bottlenecks
