Lab Notes: Which Compiler Is Fastest?

We test the latest C++ compilers from Borland, Intel, Microsoft, Symantec, and Watcom.

Al Gallant

While many developers choose C++ compilers based on market share, cost, or ease of use, I believe that performance should be the dominant criterion. I recently investigated this issue while recompiling our BYTEmark CPU benchmark on the latest versions of five well-known C++ compilers. The results uncovered surprising differences in how each compiler handles BYTEmark's CPU-intensive kernel code.

I worked with compilers from Borland International (version 5.02), Intel (2.4), Microsoft (5.0), Symantec (7.5), and Watcom (11), using the following options on each: fastest possible code, 4-byte word alignment, Pentium code generation, and Pentium register base calling. The test systems were a Dell 90-MHz Pentium, two Gateway 200-MHz Pentiums (one with MMX), a Compaq 266-MHz Pentium II, and a Gateway 300-MHz Pentium II, each running Windows 95. The reference BYTEmark test is version 2.0, released in 1995 and compiled with the Watcom version 10 C++ compiler. (To recreate the tests, look for the BYTEmark source code and documentation at http://www.byte.com/bmark/bmark.htm .)

Microsoft was the fastest in both the BYTEmark integer index and the floating-point (FP) index, yet it had somewhat lower-than-expected results in the International Data Encryption Algorithm (IDEA) block-cipher t est, which is a measure of raw speed. Microsoft admitted that the code created by the compiler generates two unnecessary memory references and promised to fix the bug in version 6.0.

The Intel compiler was next-fastest in overall performance. But it, too, did poorly in one test: Huffman text and graphics compression. Intel found that the Pentium register base calling option generates an Integer Multiply (IMUL) instruction that takes 10 clock cycles on Pentium processors, compared to only three cycles on Pentium IIs and Pentium Pros. Intel says it is less efficient to use IMUL instead of Load Effective Address (LEA, an instruction that loads a specified register with the offset of a memory location) to multiply by a constant. The company said it will consider modifying the next version of the compiler.

Borland had good integer but poor FP performance. Borland says the compiler's math libraries appear not to be optimized for Pentium code generation. The company promises a fix in the next release.

Symantec's compiler lacked the Pentium register base calling option, so we couldn't duplicate the switch settings of the other compilers. Symantec's integer numbers were good, but FP was poor. Symantec could not provide a reason.

Watcom's version 11 compiler has a known bug involving integer operations (see the August 1997 BYTE, page 23). The company provided us with a patch that, unfortunately, did not fix everything, leaving version 11 still performing slower than version 10 in some tests. Watcom promised a complete patch by the time this article is in print.

Microsoft C++ is the best performer not just because of its faster results but because of its consistency across Intel Pentium platforms. It's the smart choice for developers who want their products to have comparable performance on a wide range of Wintel systems.

---

C++ Compilers Go Head-to-Head

illustration_li nk (18 Kbytes)

Microsoft outperforms the rest in overall floating-point and integer performance, but underperforms on the IDEA encryption test.

---

Al Gallant ( ) is technical manager of the BYTE Lab.