Cache Coherency: Now More than Ever

By Steven J. Vaughan-Nichols

June 2, 2003

(Cache Coherency: Now More than Ever :  Page 1 of 1 )

Caching used to be so simple. You used caching on uniprocessor, single user systems to free yourself from the prison of slow I/O, whether that I/O was from the system bus to the processor or from the hard drive to the bus. As multiple-CPU systems, multi-user systems and the Web became commonplace, simply placing fast RAM between system components with varying I/O throughputs was no longer enough. System designers had to make sure that the available data to the processor was the "real" data, and so it was that the almost intractable problems of cache coherency emerged.

Cache Coherency Basics

Over the last few years, cache coherency theory and practice has become more critical than ever. With processors far outpacing the bus in terms of speed, and with the rapid rise of Symmetric Multiprocessing (SMP) operating systems like Windows 2000 and 2003 Server, Solaris, and Linux—not to mention the growing popularity of clustering systems such as Scyld Beowulf and MOSIX—memory and disk access has become the bottleneck in high performance computing.

So it is that now, more then ever, system designers must deal with the fundamental problem of cache coherency: displaying and writing current data without error. Cache coherency theory and practice offers many options to handle these problems, but none of them are perfect.

The core problem with reading data is deciding when to kick some data out of the cache and what should be thrown out. The most common solution is to use a least-recently-used algorithm to decide which data to exile. However, this isn't fool- or algorithm-proof.

More polished caching software and firmware use a least-frequently-used algorithm. At the price of memory and processing overhead, these schemes ensure that frequently accessed data will stay in the cache even if it hasn't been called on recently.

Cache Writing

Another, probably even more important, issue is how a cache contends with data writing. Most caches will check, using a variety of algorithms, to see if the write request will force the cache to waste time overwriting identical data that has already been written.

 Page 1 of 1