For most applications, the video adapter is the biggest bottleneck in system performance. For example, a modern hard disk can load a 3-D bar chart in 40 milliseconds, while a fast video adapter needs about 700 ms to display the same image.
To create a comprehensive series of tests to identify the graphics accelerators that can shorten the performance bottleneck, we first sought to identify the most important markets for video adapters. We concentrated on four PC applications and the high end of Macintosh graphics, as outlined below.
General-purpose. 1024 by 768 resolution with 256 colors; for general-business applications and mainstream Windows users.
Direct color. 1024 by 768 resolution with 64,000 colors; for high-end Windows graphics applications, graphic designers, and multimedia producers.
Desktop publishing. 1024
by 768 resolution with 16.7 million colors; for desktop publishers and graphics illustrators creating color publications.
CAD/CAM. 1280 by 1024 resolution with 256 colors; for engineers, architects, and draftspersons.
Macintosh graphics. 1152 by 870 resolution with 16.7 million colors; for desktop publishers and graphics illustration.
We required test boards for the PC platform to support a minimum of 1024 by 768 pixels with 256 colors in noninterlaced mode. Boards had to have a minimum of 1 MB of memory, but if an adapter could support higher amounts of RAM, we asked vendors to supply the greater amount, up to 4 MB.
Beyond the scope of this roundup are video adapters with coprocessors, such as the Texas Instruments 34020 chip. We tested a board in each resolution and at a color level that the board supported in noninterlaced mode. Because of screen flicker, we don't recommend using any board in interlaced mode for a prolonged period of time.
On the Macintosh, we concentrate
d our tests exclusively on 24-bit, true-color adapters capable of displaying 16.7 million colors on monitors up to 21 inches in size. We limited our testing to this class primarily because most Macintosh motherboards are equipped with excellent video capabilities for less-demanding applications.
Performance was our primary criterion for selecting winners. After we chose the top performers in each category, we ranked the winners and the runners-up by considering the cost, support options, usability, and any unique features the boards offered. Because of differences in retail and street prices, we considered a 15 percent cost difference to be insignificant.
In ease-of-use scores, adapters received a rating of "good" if an average user could install them without referring to the manual. An "excellent" ranking was reserved for those adapters that had exceptionally clear and complete documentation and installation software. Boards that were rated "fair" required you to consult the documentation, whil
e boards receiving a rating of "poor" needed a variety of jumper changes and/or a call to the company's technical support to configure them correctly.
Although usability included how easy a card was to install, this judgment was tempered somewhat by the fact that installation is typically a one-time task. Once you get even the most troublesome board up and running, you're likely to be concerned only about performance for the rest of that board's life.
We also considered any unique utilities or hardware features that gave a board an advantage in any of our application rankings. For example, the Matrox MGA Impression/3/V offers mode switching, which lets CAD designers display 24-bit output with 8 bits per pixel and then switch dynamically to true 24-bit color for 3-D rendering. This feature and the board's CAD-specific documentation helped the Impression/3/V rise to the top in our VL-Bus CAD/CAM evaluations. Finally, we did not consider adapters for honors unless they could complete the entire tes
t without compatibility problems, and they had to produce a clear, stable picture.
We also gave higher marks to boards that didn't require a separate VGA adapter, which not only slows the installation process but takes up an additional slot in your system. Finally, boards received higher marks if you could reach the company's support staff through a variety of methods, including a company BBS, a toll-free phone number, and a fax number.
PERFORMANCE
When writing our performance tests, we placed the most emphasis on producing tests that were a meaningful reflection of real-world conditions. To reproduce the performance of graphics applications, we designed our tests using images produced from CorelDraw, Corel Presents, Microsoft Excel, and Microsoft Word for Windows. In all, we required each board to display 15 test screens, ranging from straight text, to 2-D and 3-D bar charts, to complex full-color drawings. See the box at right for test samples.
We had hoped to use CorelDraw and Cor
el Presents as cross-platform applications for both our Windows and Macintosh tests, but delays in QuickDraw GX sidelined CorelDraw for Macintosh. Instead we used Corel's export filters to convert our drawings into PICT format. We also incorporated images from the Macintosh versions of Microsoft Word, Microsoft Excel, and Aldus Persuasion. (Because the pictures we used were specific to the platform, you should avoid making generalizations about Mac performance versus PC performance on the basis of these test results.)
In addition to mirroring real-world demands, we also designed our tests to be "cheat proof." Some graphics benchmarks, for example, use profiling to define real-world usage; however, the tests consist simply of lines being drawn on top of each other. A clever writer of video drivers can improve speeds in such tests by adjusting the driver to draw the first line and ignore the rest.
To avoid this problem, we used full application screens exactly as they are produced by applications.
To further increase our test accuracy, we used microsecond resolution timing. This allowed us to accurately measure a single screen paint, and it avoids the problem of drawing the same screen repeatedly (which is unrealistic and easy to "optimize away" in the driver).
Our Windows test software draws each of the 15 application screens into both system memory and video memory using four different color modes for more than 120 tests in all. We also measured the time it took to refresh the screen from an image cached in memory at screen depths of 1, 2, 4, 8, 16, and 32 bits per pixel. (Well-written applications will cache display images in system memory whenever possible to improve response times.)
To reach an overall response time, we averaged test results using weights derived from profiling typical Windows usage. The overall response time is the average time needed to repaint the entire screen. Use this number to gauge the performance of boards that support resolutions and color levels appropria
te to your applications.
Our Macintosh tests performed similarly, except that we tested only in 24-bit-color mode, and drawing in memory was not a consideration because this relies only on the Macintosh Toolbox code and not on the video adapter.
TEST-BED
We tested the ISA and EISA boards in a 66-MHz 486DX2 Compaq Deskpro 66M with 8 MB of memory. For the VL-Bus boards, we used DEC's MTE 486/66 with 16 MB of memory. Technicians conducted all tests using Microsoft Windows 3.1. Macintosh testing was done on a Mac Quadra 840AV with 16 MB of RAM.
Photograph: Jim Hurd (top photo, left) developed the graphics tests for this issue. Here, he and technical analyst Siva Kumar analyze test results. Chandrika Krishnamurthy and Mark Paxson (seated, bottom photo) discuss graphics accelerator board rankings with Alan Joch.
Illustration: Test Samples
This Microsoft Excel bar chart helped represent business graphics.
A CorelDraw image helped gauge color graphics speed.
This
world map tested speeds for complex monochrome drawings.
Jim Hurd, Vice President of Research and Development/NSTL, wrote the graphics benchmarks for this report. He has developed numerous tests for hardware and software during the last 10 years.
Helen Holzbaur, Project Manager/NSTL, was a network manager and systems administrator at Temple University for 10 years before joining NSTL.
Alan Joch, Senior Editor/BYTE, coordinates the combined testing between the BYTE Lab and NSTL.
Chandrika Krishnamurthy, Technical Analyst/NSTL, evaluates peripherals and systems.
Siva Kumar, Technical Analyst/NSTL, specializes in hardware and network operating-systems testing.
Mark Paxson, Manager of Design-Verification Testing Services/NSTL, specializes in hardware compatibility testing.
How We Tested
