Our performance tests subjected the modems to eight kinds of data transfer tasks and seven telephone-line conditions. We used DOS and Windows platforms to drive data through the modems.
THROUGHPUT TESTS
Our throughput tests measure each modem's speed when transmitting data in one direction and when simultaneously transmitting and receiving files. We used four different files: compressed, graphics, text, and database. These files have potential V.42bis compression ratios of 1 to 1, 2 to 1, 3 to 1, and 4 to 1, respectively. Test files ranged in size from 131 to 333 KB.
In this roundup, we report data-throughput (as well as impaired-line performance) tests in bytes per second, versus the bits-per-second ratings that are used to distinguish classes of modems (e.g., V.32terbo's 19.2 Kbps). We believe bytes per second is a clearer unit of
measure because in real-world use, modems transfer files, either compressed or not, that conventionally are measured in bytes.
We connected like pairs of modems to a TAS Series II modem tester, which can re-create almost any line condition that you may encounter throughout the world. For throughput testing, the TAS system simulated the central-office impairment conditions (i.e., line 17c 1 1). We ran the throughput tests over a telephone line with minimal impairments to represent the majority of calls made in the U.S.
We connected the modems via the TAS system to a Compaq Deskpro 66M equipped with a Hayes ESP board to ensure that communication between the TAS system and the modems utilized 16550 UARTs. We configured the modems to receive data from the computer at the fastest rate they supported (up to 115.2 Kbps).
We initialized the error-correction and data-compression engines on all the modems because the throughput tests measure the performance of these two features. Each tested modem
supported V.42 error correction and V.42bis data compression. We enabled V.42bis data compression and V.42 error correction, even if the modem's default settings specified other protocols. We also configured all the modems to use hardware (RTS/CTS) flow control. For data-compression and error-correction parameters, we used the default window and dictionary sizes.
To measure one-way transmission, modem A calls modem B, sends a file, and hangs up. Modem A repeats the process three more times. During a one-way transmission, modem B only receives data; it does not send anything back to modem A. In our two-way tests, modem A still makes four calls to modem B, but when modem B answers, both modems simultaneously send files to each other.
IMPAIRED LINE
The impairment combinations we used are based on the working papers of the EIA/TIA's (Electronic Industries Association/Telecommunications Industry Association) TR-30.3 committee. Our test conditions are a subset of the lines discussed in that comm
ittee's PM-3064 draft recommendation for network simulation for modem testing. We chose the subset to represent three broad types of lines under two different local-loop conditions. These lines introduced a variety of impairments, including long satellite delays, phase roll, and noise. (See the text box at the right for details about individual lines.)
The TAS system simulated the trunk line and local loops at both ends of the connection. Long local loops (the connection between your telephone and the telephone company's switch) do exist in the U.S. (Our long local loop simulated a telephone connection about 30,000 feet from the switch, with four loading coils.) Few of the modems had difficulty with the long local loop in itself; problems were more likely to appear when the long local loop was combined with satellite delays or noise impairments.
The satellite-delay line was quite difficult for many of the modems, but it's a condition you are not likely to encounter in the U.S. We included it bec
ause you may see a delay of that magnitude if you have worldwide business dealings.
Each pair of modems performed a one-way transfer of a 131-KB compressed file over the given simulated line at least 10 times. We recorded the times and took an average. If a modem could not complete the test over a given line, and the vendor was not able to solve the problem, we assigned a zero value.
The modems make the connection at the highest speed possible. But they do not necessarily maintain that speed. We configured them to "fall back" to a slower speed when they encountered an impairment on the line that could challenge data integrity. When the impairment disappeared, the modems could "fall forward" to the higher speed. We found that V.34 modems often fall back below the V.34 standard to 14.4 Kbps to maintain a connection and fall forward to 28.8 Kbps when the line improves.
To determine our overall performance scores when ranking modems, we weighed results of the one-way data throughput tests (60
percent), the two-way tests (20 percent), and the impaired-line scores (20 percent).
FAX TESTS
Transmission time is not really an issue for fax modems, because most of the fax machines receive data at only 9600 bps and must compress according to the CCITT standard. So, unless you know you are sending data from fax modem to fax modem, you cannot expect to see 14.4- or 19.2-Kbps performance (and if that is the case, you would see better results transmitting the file as data, rather than sending a fax).
Each modem that had fax capability was set up to send a fax to five different fax machines. We used Microsoft Windows for Workgroups to send a single-page fax to a Brother Intellifax AX-600, a Canon Faxphone 40, a Panasonic KX-F150, a Panasonic Panafax PX150, and a Sharp FO-510, which represent a real-world mix of vendors and old and new generations of fax machines. The modems were scored on the percentage of faxes they successfully sent.
INTEROPERABILITY
We set up each modem to ca
ll and answer all the other modems in the test sample. The numbers we report are the percentage of modems the test modem connected with at either 28.8 or 19.2 Kbps (i.e., the highest speed of the slowest modem). In all cases, test modems successfully connected with others in the sample. Consequently, a score of "0" is indicative only of faster-than-14.4-Kbps performance. Each of the four modems that received this score used proprietary chip sets; the overall interconnectivity of each of these modems was at least 65 percent.
FEATURES AND EASE OF USE
The features we valued most in our scoring included the protocols supported, the warranty, the number of interface indicators, the ability to spoof (i.e., emulate other handshaking protocols), support for leased lines, and auto-baud support. For fax modems, we also looked for the number of protocols and the speeds that were supported.
Contributors
Helen E. Holzbaur, Project Manager/NSTL, was a network manager and systems administrator at Templ
e University for 10 years before joining NSTL.
Alan Joch, Senior Editor/BYTE, coordinates the combined testing between the BYTE Lab and NSTL.
Siva Kumar, Senior Tester/NSTL, specializes in networking and communications hardware evaluations.
Illustration: Modem Test-Bed
Test modems are identical pairs configured for V.42 (error correction) and V.42bix (data compression)
How We Tested
