Ehcache 1.6 2 Orders of Magnitude Faster

I have been waiting for enough people to move to Java 5 to mandate it as a minimum standard for ehcache. At JavaOne 2008 I found out that a lot of people were still to make the move. Now that we are in 2009 I have decided to move to Java 5. As part of this I have done a general cleanup of the core. I can now retire backport-concurrent which has served the project well (thanks guys) and other dependencies. Ehcache-1.6 core has no dependencies.

I decided that with the improvements in concurrency support that have come along, it was time to move beyond the use of synchronized. Years ago I adopted striped locking on BlockingCache which gave amazing results but I left the core pretty much as it was. The rework adopts some new goodness in Java 5 such as CopyOnWriteArray and ConcurrentHashMap. Having said that there is nothing in Java 5 for eviction, so the new work relies heavily on some excellent contributions to provide performance for caching application that is not available in Java 5.

On my own concurrency tests, which use 70 threads simulating a typical load against a single cache, I get the following improvements in ehcache-1.6 over ehcache-1.5. (Note 70 are just for that cache. Ehcache typically has many caches, so this translates to a production system with thousands of threads against all caches)

Operation	Number of Times Faster Than Ehcache-1.5.0
get	92.5 times faster
put	30 times faster
remove	48 times faster
removeAll	80 times faster
keySet	30 times faster

Manik Surtani maintains a cache performance benchmark tool. Using that I have added ehcache-1.6. It shows dramatically the performance increases in Ehcache-1.6.

For those with less than perfect eyesight, the second column, which is too short to even have its time printed, is the ehcache-1.6 performance.

What these charts are saying, is that an ehcache, with 25 concurrent threads, is now much faster than it was. The single threaded case it no faster. But caches are not about single threads.

Now, in case everyone gets preoccupied on the comparsions between Java caches, here is an old Ehcache versus Memcached chart.

If I redid this chart using ehcache, the barely visible columns for ehcache would completely disappear on this scale.

So what is this really saying? An in-process cache, which uses a few tens of CPU operations to access data already held in memory, is much, much, much faster than going out over the network for some data, regardless of how slick the server implementation at the other end is.

But I recognise that Memcached is about a different type of caching: massive partitioned caches. The Ehcache project has the Ehcache Server for that, with RESTful and SOAP APIs. The RESTful implementation uses a variety of tricks such as conditional get, the ability to have hardware and software load balancers (think ngnx) perform URI routing, head, HTTP1.1 compression and pipelining plus the goodness of modern NIO Java Web Containers to seriously give memcached a run for its money. I will be doing some performance comparisons between Memcached and Ehcache Server in the near future.

What else is next? The above numbers are for MemoryStore based caches. I am also going to give the DiskStore a work over, with lots of suggestions made to me in the last year. Stay tuned.