JHAT

 

http://jovial.com/hat/doc/README.html

 

HAT -- The Java Heap Analysis Tool

Bill Foote
bill.foote at sun d o t com, billf at jovial d o t com

Introduction

This document describes a heap dump file for Java programs, and a tool to analyze these files. This tool can help you to debug an analyze the objects in a running Java program. It is particularly useful when debugging unintentional object retention.

Unintentional object retention, a phenomenon sometimes called "memory leaks," can come about in several ways. I call it a unintentional object retention when an object that is no longer needed is referenced through some path from the rootset. This can happen, for example, if an unintentional static reference to an object remains after the object is no longer needed, if an Observer or Listener fails to de-register itself from its subject when it is no longer needed, or if a Thread that refers to an object does not terminate when it should.

This tool helps to debug unnecessary object retention by providing a convenient means to browse the object topology in a heap snapshot that is generated by the Java VM. This shapshot is in a JDK 1.2 heap profile file (generated with -Xrunhprof). The tool allows a number of queries, including "show me all reference paths from the rootset to this object". It is this query that is most useful for finding unnecessary object retention.

This document will discuss each of the queries that the analysis tool offers.

Donations to Charities

HAT was done mostly nights and weekends, starting when I was at JavaSoft. To cover Sun there is a license, but as the author may I make a small request? Could you please read the words I've written on the charitable donations page, and do as you feel appropriate? Thank you.

Generating a .hprof File and Running the Program

To generate a .hprof file, you should run java with the -Xrunhprof flag. The format recognized by this program first appeared in JDK 1.2 beta3. HAT must be given a binary hprof file. This is done with the "format=b" modifier to the -Xrunhprof command. For example, to run the program Main and generate a binary heap profile in the file dump.hprof, use the following command:

    java -Xrunhprof:file=dump.hprof,format=b Main

As of JDK 1.2, you can add a heap dump to a .hprof file with control-backslash under Unix or OS/X, or control-break in Win32. Some sample .hprof files are provided in the hprof_files subdirectory (doc/hprof_files). They were produced with a small slot car animation program I wrote in a former life. The source for that program can be found under http://jovial.com/slotCar/.

To run the hat server program, use the run shell script or batch file provided in the bin directory. You'll probably want to copy it and hat.jar somewhere sensible (I use ~/bin on Unix and OS/X, and c:/bin on Windows). Once that's done, to analyze a slot car hprof file, you can do the following:

	cd doc/hprof_files
	hat -port=7002 slot_car_jdk_1_5_0_beta2.hprof

If a port is not specified, it defaults to 7000.

The server can compare two heap dumps, and it's possible to specify a numbered heap dump in the file. For example, to compare the first and second heap dump in slotCar.hprof, use this command:

	hat -baseline slot_car_from_jdk_1_2_beta3.hprof#1 slot_car_from_jdk_1_2_beta3.hprof#2

You can specify a file that influences the reachable objects query. This file lists the data members that should be excluded when generating this query. This can be used to determined the size of a given object and all objects reachable from it, excluding observers and other objects that aren't "owned" by the target object. To specify a set of data members to exclude, use the -exclude command line option:

	./hat -exclude slotCar.exclude slot_car_from_jdk_1_2_beta3.hprof

This file should contain the fully-qualified name of each data member to exclude, with a newline between each data member. If this file is modified while a server is running, it will be re-read.

The program sets itself up as an http server on the port you specify. You access it by bringing up the default page in the browser of your choice. For example, if you're running hat on the same computer as your browser, you can probably call it "localhost", if not "127.0.0.1" will do. Assuming your OS supports calling it localhost, you could access the server started above by navigating to the URL "http://localhost:7002/". From there, you can run other queries by clicking on hyperlinks, or by directly typing in URLs.

This document contains links to sample HTML pages generated by the server. The links within these sample pages will not work. To generate a working example, run the server program with the provided .hprof file.

Note on JDK versions

The feature of generating a .hprof file was considered experimental in JDK 1.2 through 1.4. It worked well with the classic VM on JDK 1.2, but some VMs and some versions of the JDK might not produce .hprof files compatible with HAT. As of JDK 1.5, .hprof files are produced by an experimental agent that connects to the VM via a supported profiling interface. As such, it's more stable and dependable, though I'm told there were some issues with version prior to update 3. I recommend using a JDK version of 1.5 update 3 or better with HAT.

Sample HAT queries

The All Classes Query

Example: http://localhost:7002/ 
Example: http://localhost:7002/allClassesWithPlatform/

The default page shows you all of classes present on the heap, excluding platform classes. This list is sorted by fully-qualified class name, and broken out by package. By clicking on the name of a class, you are taken to the Class query. At the bottom of the page is a link you can click to be taken to other queries.

The second variant of this query includes the platform classes. Platform classes include classes whose fully-qualified names start with things like "java.", "sun.", "javax.swing.", or "char[". The list of prefixes is in a system resource file called "resources/platform_names.txt". If you want to override this list, just replace it in the jar file, or arrange for your replacement to occur first on the classpath when hat is invoked.

The Class Query

Example: http://localhost:7002/class/jovial.slotCar.Car

The Class query shows you information about a class. This includes its superclass, any subclasses, instance data members, and static data members. From this page you can navigate to any of the classes that are referenced, or you can navigate to an Instances query.

The Instances Query

Example: http://localhost:7002/instances/jovial.slotCar.Car
Example: http://localhost:7002/allInstances/jovial.slotCar.animator.Drawable

The instances query shows you all instances of a given class. The "allInstances" variant includes instances of subclasses of the given class as well. From here, you can navigate back to the source class, or you can navigate to an Object query on one of the instances.

The Object Query

Example: http://localhost:7002/object/ee3077c8

The object query gives you information about an object that was on the heap. From here, you can navigate to the class of the object, to the value of any data members of the object. You can also navigate to objects that refer to this object. Perhaps the most valuable query is at the end: The Roots query ("Reference Chains from Rootset").

Note that the object query also gives you a stack backtrace of the point of allocation of the object.

The Roots Query

Example: http://localhost:7002/roots/ee3077c8
Example: http://localhost:7002/allRoots/ee3077c8

The roots query gives you reference chains from the rootset to a given object. It will give you one chain for each member of the rootset from which the given object is reachable. When calculating these chains, it does a depth-first search, so that it will provide reference chains of minimal length.

There are two kinds of roots query: One that excludes weak references ("roots"), and one that includes them ("allRoots"). A weak reference is a reference object that does not prevent its referent from being made finalizable, finalized, and then reclaimed. Weak references are subclasses of sun.misc.Ref, or java.lang.ref.WeakReference as of JDK 1.2. If an object is only referred to by a weak reference, it usually isn't considered to be retained, because the garbage collector can collect it as soon as it needs the space.

The server sorts rootset reference chains by the type of the root, in this order:

• Static data members of Java classes.
• Java local variables. For these roots, the thread responsible for them is shown. Because a Thread is a Java object, this link is clickable. This allows you, for example, to easily navigate to the name of the thread.
• Native static values.
• Native local variables. Again, such roots are identified with their thread.

This is probably the most valuable query in HAT, if you're debugging unintentional object retention. Once you find an object that's being detained, this query tells you why it's being retained.

The Reachable Objects Query

Example: http://localhost:7002/reachableFrom/ee3077c8

This query, accessible from the Object query, shows the transitive closure of all objects reachable from a given object. This list is sorted in decreasing size, and alphabetically within each size. At the end, the total size of all of the reachable objects is given. This can be useful for determining the total runtime footprint of an object in memory, at least in systems with simple object topologies.

This query is most valuable when used in conjunction with the "-exclude" command line option, described above. This is useful, for example, if the object being analyzed is an Observable. By default, all of its Observers would be reachable, which would count against the total size. -exclude allows you to exclude the data members java.util.Observable.obs and java.util.Observable.arr.

The Instance Counts for All Classes Query

Example: http://localhost:7002/showInstanceCounts/ 
Example: http://localhost:7002/showInstanceCounts/includePlatform/

This query shows the count of instances for every class in the system, excluding platform classes. It is sorted in descending order, by instance count. A good way to spot a problem with unintentional object retention is to run a program for a long time with a variety of input, then request a heap dump. Looking at the instance counts for all classes, often a class you know will jump out at you, because there are more than you expect. Then, you can analyze them to determine why they're being retained (possibly using the roots query).

There's also a variant of this that includes platform classes. See the notes under the All Classes query for notes about what is considered to be a platform class.

The All Roots Query

Example: http://localhost:7002/showRoots/

This query shows all members of the rootset.

The New Instances Query

Example: http://localhost:7002/newInstances/java.awt.Point
Example: http://localhost:7002/allNewInstances/java.lang.Object

The New Instances query is only available if you invoke the hat server with two heap dumps. It is like the Instances query, but it only shows "new" instances. An instance is considered new if it is in the second heap dump, and there is no object of the same type with the same ID in the baseline heap dump. An object's ID is a 32 bit integer that uniquely identifies the object, and it is assigned by the VM. In the VM implementations that were current when HAT was written (JDK 1.1.x and betas of JDK 1.2), an object's ID was the address of its handle, so it was pretty stable. Because handle address could be re-used, the new instances query could give inaccurate results. In more modern VM implementations, the object ID could well be less table, and the results might be unusable. With betas of JDK 1.2, comparing snapshots taken within a relatively short interval, the results were pretty good. YMMV.

Summary

The hat server can be valuable for debugging and understanding programs. It allows you to navigate object structures to learn how objects are interconnected in a program. It also allows you to trace the references to a given object from the rootset, which can be valuable for tracking down unnecessary object retention. We found the hat server to be a valuable debugging aid, and hope that you will, too.

For more information, please consult the HAT page at http://hat.dev.java.net/.

MAIN PAGE |  SENDING COMMENTS |  DONATIONS |  HISTORY