Java thread dump的分析思路

Java thread dump的分析思路：
第一步：java thread dump中首先需要检查是否有死锁，即deadlock，通过jstack生成的dump，
会自动将deadlock列出来；如果有deadlock，需要查看引起死锁的代码；

第二步：检查是否有条件等待状态的线程【wait on condition】，如果有，需要检查对应代码的业务逻辑是否合理！

第三步：其次要检查是否大量线程发生block，一般blocked thread的状态有【waiting for monitor entry】和【in Object.wait()】
前者是在Entry Set队列中，【in Object.wait()】是在Wait Set队列中。

第四步：检查是否发生热锁，发生热锁的必要条件是：
（1）、cpu繁忙并且系统态远大于用户态
（2）、cpu数目越多，性能反而越差，可以通过taskset指定引用哪些cpu核。
如果发生热锁，需要定位到代码段，分析其代码逻辑并进行改进。


注意：1、在分析thread dump时，要比对至少三次的thread dump，而不要拿某一个dump来说事，因为thread dump是某个时刻的快照
，需要横向比对定位问题。2、NIO比BIO在多线程处理上更好一些。3、代码中尽量少用synchronized关键字，除非你对多线程管理的很好。

一般线程的状态有以下几种：
************************
1.线程状态：【runnable】
"Running HTTPHandler-995" prio=10 tid=0x2eff0800 nid=0x108a runnable [0x7c75e000]
   java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:146)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
- locked <0x50724c18> (a java.io.BufferedInputStream)
at com.cnd.web.http.tcp.TCP_BIOProtocol.readLine(Unknown Source)
at com.cnd.web.http.tcp.TCP_BIOProtocol.readRequestLine(Unknown Source)
at com.cnd.web.http.tcp.TCP_BIOProtocol.read(Unknown Source)
at com.cnd.web.http.tcp.TCP_BIOConnection.getNextRequest(Unknown Source)
at com.cnd.web.http.ConnectionHandler.processConnection(Unknown Source)
at com.cnd.web.http.ConnectionHandler.run(Unknown Source)
at com.cnd.util.ThreadPoolImpl$WorkerThread.run(Unknown Source)
************************
2.线程状态：【wait on condition】

*************************
3.线程状态：【waiting for monitor entry】
"Running ORBWorker-299" prio=10 tid=0x2ea82400 nid=0x10a3 waiting for monitor entry [0x7cc2c000]
   java.lang.Thread.State: BLOCKED (on object monitor)
at com.cnd.corba.ee.impl.orb.ORBImpl.create_any(Unknown Source)
- locked <0x4632e838> (a com.cnd.corba.ee.impl.orb.ORBImpl)
at com.cnd.corba.ee.impl.javax.rmi.CORBA.Util.writeAny(Unknown Source)
at com.cnd.corba.rmi.Util.writeAny(Unknown Source)
at javax.rmi.CORBA.Util.writeAny(Util.java:96)
at com.cnd.corba.ee.impl.io.IIOPOutputStream.writeObjectField(Unknown Source)
at com.cnd.corba.ee.impl.io.IIOPOutputStream.writeField(Unknown Source)
at com.cnd.corba.ee.impl.io.OutputStreamHook$HookPutFields.write(Unknown Source)
at com.cnd.corba.ee.impl.io.OutputStreamHook.writeFields(Unknown Source)
at java.beans.PropertyChangeSupport.writeObject(PropertyChangeSupport.java:418)
at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at com.cnd.corba.ee.impl.io.IIOPOutputStream.invokeObjectWriter(Unknown Source)
at com.cnd.corba.ee.impl.io.IIOPOutputStream.outputObject(Unknown Source)
at com.cnd.corba.ee.impl.io.IIOPOutputStream.simpleWriteObject(Unknown Source)
at com.cnd.corba.ee.impl.io.ValueHandlerImpl.writeValueInternal(Unknown Source)
at com.cnd.corba.ee.impl.io.ValueHandlerImpl.writeValueWithVersion(Unknown Source)
at com.cnd.corba.ee.impl.io.ValueHandlerImpl.writeValue(Unknown Source)
at com.cnd.corba.ee.impl.encoding.CDROutputStream_1_0.writeRMIIIOPValueType(Unknown Source)
at com.cnd.corba.ee.impl.encoding.CDROutputStream_1_0.write_value(Unknown Source)
at com.cnd.corba.ee.impl.encoding.CDROutputStream_1_0.write_value(Unknown Source)
at com.cnd.corba.ee.impl.encoding.CDROutputStream_1_0.write_value(Unknown Source)
at com.cnd.corba.ee.impl.encoding.CDROutputStream.write_value(Unknown Source)
at com.cnd.corba.ee.impl.io.IIOPOutputStream.writeObjectField(Unknown Source)
at com.cnd.corba.ee.impl.io.IIOPOutputStream.outputClassFields(Unknown Source)
at com.cnd.corba.ee.impl.io.IIOPOutputStream.defaultWriteObjectDelegate(Unknown Source)
at com.cnd.corba.ee.impl.io.IIOPOutputStream.outputObject(Unknown Source)
at com.cnd.corba.ee.impl.io.IIOPOutputStream.simpleWriteObject(Unknown Source)
at com.cnd.corba.ee.impl.io.ValueHandlerImpl.writeValueInternal(Unknown Source)
at com.cnd.corba.ee.impl.io.ValueHandlerImpl.writeValueWithVersion(Unknown Source)
at com.cnd.corba.ee.impl.io.ValueHandlerImpl.writeValue(Unknown Source)
at com.cnd.corba.ee.impl.encoding.CDROutputStream_1_0.writeRMIIIOPValueType(Unknown Source)
at com.cnd.corba.ee.impl.encoding.CDROutputStream_1_0.write_value(Unknown Source)
at com.cnd.corba.ee.impl.encoding.CDROutputStream_1_0.write_value(Unknown Source)
at com.cnd.corba.ee.impl.encoding.CDROutputStream_1_0.write_abstract_interface(Unknown Source)
at com.cnd.corba.ee.impl.encoding.CDROutputStream.write_abstract_interface(Unknown Source)
at com.cnd.corba.ee.impl.javax.rmi.CORBA.Util.writeAbstractObject(Unknown Source)
at com.cnd.corba.rmi.Util.writeAbstractObject(Unknown Source)
at javax.rmi.CORBA.Util.writeAbstractObject(Util.java:147)
at com.cnd.corba.ee.impl.io.IIOPOutputStream.writeObjectOverride(Unknown Source)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:339)
at java.util.ArrayList.writeObject(ArrayList.java:673)
at sun.reflect.GeneratedMethodAccessor43.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at com.cnd.corba.ee.impl.io.IIOPOutputStream.invokeObjectWriter(Unknown Source)
at com.cnd.corba.ee.impl.io.IIOPOutputStream.outputObject(Unknown Source)
at com.cnd.corba.ee.impl.io.IIOPOutputStream.simpleWriteObject(Unknown Source)
at com.cnd.corba.ee.impl.io.ValueHandlerImpl.writeValueInternal(Unknown Source)
at com.cnd.corba.ee.impl.io.ValueHandlerImpl.writeValueWithVersion(Unknown Source)
at com.cnd.corba.ee.impl.io.ValueHandlerImpl.writeValue(Unknown Source)
at com.cnd.corba.ee.impl.encoding.CDROutputStream_1_0.writeRMIIIOPValueType(Unknown Source)
at com.cnd.corba.ee.impl.encoding.CDROutputStream_1_0.write_value(Unknown Source)
at com.cnd.corba.ee.impl.encoding.CDROutputStream_1_0.write_value(Unknown Source)
at com.cnd.corba.ee.impl.encoding.CDROutputStream_1_0.write_value(Unknown Source)
at com.cnd.corba.ee.impl.encoding.CDROutputStream.write_value(Unknown Source)
at com.cnd.corba.ee.impl.io.IIOPOutputStream.writeObjectField(Unknown Source)
at com.cnd.corba.ee.impl.io.IIOPOutputStream.outputClassFields(Unknown Source)
at com.cnd.corba.ee.impl.io.IIOPOutputStream.defaultWriteObjectDelegate(Unknown Source)
at com.cnd.corba.ee.impl.io.IIOPOutputStream.outputObject(Unknown Source)
at com.cnd.corba.ee.impl.io.IIOPOutputStream.simpleWriteObject(Unknown Source)
at com.cnd.corba.ee.impl.io.ValueHandlerImpl.writeValueInternal(Unknown Source)
at com.cnd.corba.ee.impl.io.ValueHandlerImpl.writeValueWithVersion(Unknown Source)
at com.cnd.corba.ee.impl.io.ValueHandlerImpl.writeValue(Unknown Source)
at com.cnd.corba.ee.impl.encoding.CDROutputStream_1_0.writeRMIIIOPValueType(Unknown Source)
at com.cnd.corba.ee.impl.encoding.CDROutputStream_1_0.write_value(Unknown Source)
at com.cnd.corba.ee.impl.encoding.CDROutputStream_1_0.write_value(Unknown Source)
at com.cnd.corba.ee.impl.encoding.CDROutputStream.write_value(Unknown Source)
at com.cnd.corba.ee.impl.corba.TCUtility.marshalIn(Unknown Source)
at com.cnd.corba.ee.impl.corba.AnyImpl.write_value(Unknown Source)
at com.cnd.corba.ee.impl.encoding.CDROutputStream_1_0.write_any(Unknown Source)
at com.cnd.corba.ee.impl.encoding.CDROutputStream.write_any(Unknown Source)
at com.cnd.corba.ee.impl.javax.rmi.CORBA.Util.writeAny(Unknown Source)
at com.cnd.corba.rmi.Util.writeAny(Unknown Source)
at javax.rmi.CORBA.Util.writeAny(Util.java:96)
at com.cnd.corba.ee.impl.io.IIOPOutputStream.writeObjectField(Unknown Source)
at com.cnd.corba.ee.impl.io.IIOPOutputStream.outputClassFields(Unknown Source)
at com.cnd.corba.ee.impl.io.IIOPOutputStream.defaultWriteObjectDelegate(Unknown Source)
at com.cnd.corba.ee.impl.io.IIOPOutputStream.outputObject(Unknown Source)
at com.cnd.corba.ee.impl.io.IIOPOutputStream.simpleWriteObject(Unknown Source)
at com.cnd.corba.ee.impl.io.ValueHandlerImpl.writeValueInternal(Unknown Source)
at com.cnd.corba.ee.impl.io.ValueHandlerImpl.writeValueWithVersion(Unknown Source)
at com.cnd.corba.ee.impl.io.ValueHandlerImpl.writeValue(Unknown Source)
at com.cnd.corba.ee.impl.encoding.CDROutputStream_1_0.writeRMIIIOPValueType(Unknown Source)
at com.cnd.corba.ee.impl.encoding.CDROutputStream_1_0.write_value(Unknown Source)
at com.cnd.corba.ee.impl.encoding.CDROutputStream_1_0.write_value(Unknown Source)
at com.cnd.corba.ee.impl.encoding.CDROutputStream_1_0.write_value(Unknown Source)
at com.cnd.corba.ee.impl.encoding.CDROutputStream.write_value(Unknown Source)
at com.cssnet.platform.baseframe.core.facade._DomainFacadeBean_ObjectImpl_2_Tie._invoke(Unknown Source)
at com.cnd.corba.ee.impl.protocol.CorbaServerRequestDispatcherImpl.dispatchToServant(Unknown Source)
at com.cnd.corba.ee.impl.protocol.CorbaServerRequestDispatcherImpl.dispatch(Unknown Source)
at com.cnd.corba.ee.impl.protocol.CorbaMessageMediatorImpl.handleRequestRequest(Unknown Source)
at com.cnd.corba.ee.impl.protocol.CorbaMessageMediatorImpl.handleRequest(Unknown Source)
at com.cnd.corba.ee.impl.protocol.CorbaMessageMediatorImpl.handleInput(Unknown Source)
at com.cnd.corba.ee.impl.protocol.giopmsgheaders.RequestMessage_1_2.callback(Unknown Source)
at com.cnd.corba.ee.impl.protocol.CorbaMessageMediatorImpl.handleRequest(Unknown Source)
at com.cnd.corba.ee.impl.transport.SocketOrChannelConnectionImpl.dispatch(Unknown Source)
at com.cnd.corba.ee.impl.transport.SocketOrChannelConnectionImpl.read(Unknown Source)
at com.cnd.corba.ee.impl.transport.ReaderThreadImpl.doWork(Unknown Source)
at com.cnd.corba.plugin.ee.ThreadPoolImpl$WorkAdapter.run(Unknown Source)
at com.cnd.util.ThreadPoolImpl$WorkerThread.run(Unknown Source)
************************
4.线程状态：【in Object.wait()】
"Running ORBWorker-282" prio=10 tid=0x2f194800 nid=0x107c in Object.wait() [0x7c4ee000]
   java.lang.Thread.State: BLOCKED (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x46b47480> (a java.lang.Object)
at com.cnd.corba.ee.impl.transport.SocketOrChannelConnectionImpl.writeLock(Unknown Source)
- locked <0x46b47480> (a java.lang.Object)
at com.cnd.corba.ee.impl.encoding.BufferManagerWriteStream.sendFragment(Unknown Source)
at com.cnd.corba.ee.impl.encoding.BufferManagerWriteStream.sendMessage(Unknown Source)
at com.cnd.corba.ee.impl.encoding.CDROutputObject.finishSendingMessage(Unknown Source)
at com.cnd.corba.ee.impl.protocol.CorbaMessageMediatorImpl.sendResponse(Unknown Source)
at com.cnd.corba.ee.impl.protocol.CorbaMessageMediatorImpl.handleRequest(Unknown Source)
at com.cnd.corba.ee.impl.protocol.CorbaMessageMediatorImpl.handleInput(Unknown Source)
at com.cnd.corba.ee.impl.protocol.giopmsgheaders.RequestMessage_1_2.callback(Unknown Source)
at com.cnd.corba.ee.impl.protocol.CorbaMessageMediatorImpl.handleRequest(Unknown Source)
at com.cnd.corba.ee.impl.transport.SocketOrChannelConnectionImpl.dispatch(Unknown Source)
at com.cnd.corba.ee.impl.transport.SocketOrChannelConnectionImpl.read(Unknown Source)
at com.cnd.corba.ee.impl.transport.ReaderThreadImpl.doWork(Unknown Source)
at com.cnd.corba.plugin.ee.ThreadPoolImpl$WorkAdapter.run(Unknown Source)
at com.cnd.util.ThreadPoolImpl$WorkerThread.run(Unknown Source)
***********************************
【注意】线程的6种状态：New => Runnable => Terminated => Waiting ||  Timed_wating || Blocked，死锁不是线程的状态。