Java threads may not use all your CPUs

Enterprise-level applications often need more CPU power than a single-CPU machine can provide, but getting applications to effectively use multiple CPUs can be tricky. If you're lucky, you may be able to run an application on separate single-CPU machines; Web servers scale this way. But many applications, such as databases, need to run on a single machine, which will require additional CPUs to be scaled.

In this article, we'll examine tests that show that Java threads in Sun's JRE 1.1.7 do not use more than two processors by default, regardless of how many are in the machine. Java is advertised as automatically scaling to use all available processors, but we'll see below that this is not entirely true. We'll also look at some simple ways to maximize Java multithreading performance on multi-CPU machines.

Note: My tests indicate that JRE 1.1.7 does not use more than two CPUs when running pure Java programs. Under other conditions, such as with native code that has native OS-level threads, JRE 1.1.7 may utilize all of the CPUs.

The scoop on symmetric multiprocessing

Getting multiple equivalent CPUs to cooperate in one machine is known as symmetric multiprocessing (SMP); running a single-CPU machine is known as uniprocessing. SMP machines' CPU modules usually plug into a bus. In theory, you add capacity by plugging in more CPUs.

Unfortunately, because coordinating CPU activity is difficult and most applications spend a lot of time waiting for I/O rather than using the CPU, using two processors doesn't necessarily double your throughput. Performance may even degrade with multiple CPUs. Indeed, for software not designed for an SMP system, adding a CPU may increase performance by no more than 30 percent.

For CPU-intensive programs designed for SMP, a second CPU may increase performance by 80 or 90 percent, depending on what the application is doing. CPU-intensive programs that can run as parallel processes or threads benefit more from additional CPU power than programs that spend most of their CPU time idle, waiting for a network or disk.

SMP options

One way to use multiple CPUs on a single machine is to run multiple processes that communicate via the various forms of interprocess communication, such as semaphores and shared memory. The operating system will automatically allocate different CPUs to different processes. Interprocess communication techniques are not especially portable, though. One alternative is using native thread programming in C or C++: this is difficult, but is a way to use multiple CPUs in a single process with better portability.

Java thread programming is simple by comparison and provides high portability. Java threads have also been advertised as scaling well on multi-CPU machines. You can run Java threads in parallel on multiple CPUs, but only if you use a native-threads library for Java instead of the green threads option. (The green thread implementation does all thread scheduling within the JVM. On Unix, that means all threads will run in a single process and never use multiple CPUs.)