J java.net.SocketInputStream.socketRead0(Ljava/io/FileDescriptor;[BIII)I

Bug ID:	5040096
Votes	0
Synopsis	Vtest/Vmark fail after 6 hrs run on windows2003 AMD 64bits with C2 flag
Category	java:runtime
Reported Against	tiger-beta2
Release Fixed	1.5(tiger-b57)
State	<!-- remove filtering state: show bug status/substatus <font face="" color="#ff0000">Verified</font> --> 10-Fix Delivered, Verified, <!-- show area regardless of statusDetail --> bug
Priority:	1-Very High
Related Bugs	6354368 , 4996629 , 5062728
Submit Date	29-APR-2004
Description	JDK/VM version ============= vm_info: Java HotSpot(TM) 64-Bit Server VM (1.5.0-beta2-b49) for windows-amd64, built on Apr 28 2004 01:31:31 by "java_re" with unknown MS VC++:1400 Platform ============= Windows_NT JTG-AMD2 5 customer 586 Error message ============= # An unexpected error has been detected by HotSpot Virtual Machine: # # EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000077fa0c46, pid=2572, t id=5936 # # Java VM: Java HotSpot(TM) 64-Bit Server VM (1.5.0-beta2-b49 mixed mode) # Problematic frame: # C Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) J java.net.SocketInputStream.socketRead0(Ljava/io/FileDescriptor;[BIII)I J java.net.SocketInputStream.read([BII)I J java.io.BufferedInputStream.read()I J COM.volano.e.run()V J java.lang.Thread.run()V v ~I2CAdapter v ~StubRoutines::call_stub [ntdll.dll+0xa0c46] Please telnet with root/admin and change to d:\tmp directory to check it.
Work Around	N/A
Evaluation	I have not been able to reproduce it yet. Volano mark is a total resource hog and when I run it, it fails due to various things like running out of port numbers. Looking at the stack trace, it appears to be failing in ntdll.dll rather than winsock, so it might be as a result of a problem with the malloc() in the socketRead0 routine. This code has not changed in a long time, so I doubt there is a bug in it. I will continue trying to reproduce it. xxxxx@xxxxx 2004-05-07 Ok. A Windb stack trace shows it is definitely crashing in malloc. Also, interestingly, it is not the server which crashes, rather the client which is run repeatedly as a standalone application, and which runs successfully several hundred times before the crash happens. I am pretty confident that this is not a JDK bug, but is an OS bug. malloc should never crash regardless of what parameters it is given. xxxxx@xxxxx 2004-05-11 Stack trace shown below: ntdll!NtRaiseHardError+0xa USER32!MB_GetString+0x5f7 USER32!SoftModalMessageBox+0xefc USER32!MessageBoxTimeoutA+0x17d USER32!MessageBoxA+0x50 jvm!os::message_box+0x15 jvm!VMError::show_message_box+0x89 jvm!VMError::report_and_die+0x137 jvm!topLevelExceptionFilter+0x40a ntdll!RtlQueryProcessDebugInformation+0xcbe ntdll!RtlLookupFunctionEntry+0x69 ntdll!KiUserExceptionDispatcher+0x2d ntdll!RtlQueryProcessBackTraceInformation+0xc7b9 ntdll!RtlAllocateHeap+0xe5 MSVCRT!malloc+0x3a net!Java_java_net_SocketInputStream_socketRead0+0xb6 0x7ff`bf0bc9ad 0x2`00000001 0x7ff`bf0a0904 0x7ff`b39df3a8 xxxxx@xxxxx 2004-05-11 I have run several experiments attempting to isolate the cause of this failure. I have reproduced this failure on three different builds of Windows Server2003 1068, 1073 and 1184. This bug reproduces in interpreted mode -Xint. This bug fails on B48 in addition to B49 where it was originally reported. I've attempted to use the Microsoft Debugging Utilities to performance a heap verification on each malloc / free call but since this slows down the test, it does not fail. I've also run the test on the Microsoft Checked build and it did not fail. When it does crash, I get the same stack dump in the failing thread as Michael reports. One interesting note is that the Volano client appears to be in the process of terminating its connection threads because there are only 52 threads left at the time of the crash and normally there are over 400. I've scanned the VM sources involved in thread termination and I don't see any race conditions involved in freeing memory when a GC could occur, etc. This may be a coincidence, but I've done a successful overnight run on two different systems using the switch -XX:+UseDefaultStackSize. This causes the memory for the thread stacks to be reserved and not committed. I don't understand why this would fix this problem but might be a clue for determining what is going on. I don't agree with the statement that the OS should never get a segv from within malloc. If a program corrupts the C Heap by writing garbage to heap header data structures, this would cause the crash that we are seeing. xxxxx@xxxxx 2004-05-19 I have been able to reproduce the exact crash reported here with a small C++ test case. This is a bug in Windows. Compile the small test case with "cl /MD mallocbasher.cpp" and run the test case on a multi-cpu Windows Server 2003 AMD64 box and it will crash in a few minutes. I submitted a bug on microsoft's beta web site at beta.microsoft.com. The bug number is 154243817. xxxxx@xxxxx 2004-06-07 It turns out that Microsoft expects these Access Violations. They are using structured exception handling (try/except blocks) in the malloc library and need to get control on AV's caused by their code. Since we are using Vectored Exception Handing rather than SEH, we see this AV first and report it as a fatal error. The full and correct fix for this problem is to stop using Vectored Exceptions and support SEH in compiled code. This will require significant effort/testing/risk. We must register all dynamically generated code in the VM using RtlInstallFunctionCallback or RtlAddFunctionTable APIs. Since it is late in the 1.5 release, I will put a short term fix into 1.5 which will pass AV's on from our exception handling code if the AV is generated from NTDLL.DLL. I will open up a new bug to keep track of this issue. xxxxx@xxxxx 2004-06-14
Comments	Include a link with my name & email    Submitted On 08-FEB-2007 Jodeleit We encounter this with jdk 1.6. I don't see why this is marked as "fixed"? Content of hs..pid: # # An unexpected error has been detected by Java Runtime Environment: # # EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x7c928fea, pid=3456, tid=4008 # # Java VM: Java HotSpot(TM) Server VM (1.6.0-b105 mixed mode) # Problematic frame: # C [ntdll.dll+0x18fea] # # If you would like to submit a bug report, please visit: # http://java.sun.com/webapps/bugreport/crash.jsp # --------------- T H R E A D --------------- Current thread (0x10746400): JavaThread "Thread-24" daemon [_thread_in_native, id=4008] siginfo: ExceptionCode=0xc0000005, writing address 0x00000010 Registers: EAX=0x00000000, EBX=0x00000000, ECX=0x00001338, EDX=0x0fd8682c ESP=0x12a1f034, EBP=0x12a1f0a8, ESI=0x0fd8681c, EDI=0x00000000 EIP=0x7c928fea, EFLAGS=0x00010246 Top of Stack: (sp=0x12a1f034) 0x12a1f034: 0fda1488 00000000 00000000 ffb3b4c0 0x12a1f044: ffffffff 000c2a58 00000000 00000000 0x12a1f054: 12a1f0cc 719b57c9 0000133c 00001338 0x12a1f064: 00000002 719b44b0 12a1f158 12a1f164 0x12a1f074: 00000001 12a1f140 00000001 00000000 0x12a1f084: 00000020 12a1f14c 12a1f150 00000000 0x12a1f094: 00000000 0000133c 00000000 00000000 0x12a1f0a4: 00001338 12a1f140 7c91104b 00d8681c Instructions: (pc=0x7c928fea) 0x7c928fda: 8b 46 10 3b c3 89 45 fc 0f 84 9e 00 00 00 8b 06 0x7c928fea: ff 40 10 8b 45 fc 83 e0 01 89 45 e8 8b 06 ff 40 Stack: [0x129d0000,0x12a20000), sp=0x12a1f034, free space=316k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) C [ntdll.dll+0x18fea] C [ntdll.dll+0x104b] Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) j java.net.SocketInputStream.socketRead0(Ljava/io/FileDescriptor;[BIII)I+0 j java.net.SocketInputStream.read([BII)I+84 j java.io.BufferedInputStream.fill()V+175 j java.io.BufferedInputStream.read1([BII)I+44 j java.io.BufferedInputStream.read([BII)I+49 j com.sun.jndi.ldap.Connection.run()V+30 j java.lang.Thread.run()V+11 v ~StubRoutines::call_stub Submitted On 27-FEB-2007 We are encountering this problem with jdk 1.6, too. Our application uses many sockets and actually, it is not operational with jdk 1.6 # An unexpected error has been detected by Java Runtime Environment: # # EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x7c8396d0, pid=8032, tid=2076 # Java VM: Java HotSpot(TM) Client VM (1.6.0-b105 mixed mode) # Problematic frame: # C [ntdll.dll+0x396d0] # # If you would like to submit a bug report, please visit: # http://java.sun.com/webapps/bugreport/crash.jsp # --------------- T H R E A D --------------- Current thread (0x26fa7400): JavaThread "(JSK) mux reader" daemon [_thread_in_native, id=2076] siginfo: ExceptionCode=0xc0000005, reading address 0x00000014 Registers: EAX=0x00000000, EBX=0x273476c0, ECX=0xfffffffc, EDX=0x00000000 ESP=0x287cef20, EBP=0x287cef48, ESI=0x273476bc, EDI=0x00000004 EIP=0x7c8396d0, EFLAGS=0x00010246 Top of Stack: (sp=0x287cef20) 0x287cef20: 00000001 273476c0 00000000 00000000 0x287cef30: 287ceff8 7ff75000 00000000 00000020 0x287cef40: 000022f0 00000000 287cef68 7c839620 0x287cef50: 00000000 00000004 00000000 273a15f0 0x287cef60: 00000000 00000004 287ceff8 272c446e 0x287cef70: 273476bc 287cf010 00000000 287ceff8 0x287cef80: 00000001 71b2199f ffffffff 00002194 0x287cef90: 273a15f0 00000001 00000000 272cdeb0 Instructions: (pc=0x7c8396d0) 0x7c8396c0: 8b c2 f0 0f b1 0b 3b c2 0f 85 ea 6c 00 00 8b 06 0x7c8396d0: 8b 50 14 8b 7d f8 8b 5d 08 42 89 50 14 8b c7 83 Stack: [0x28780000,0x287d0000), sp=0x287cef20, free space=315k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) C [ntdll.dll+0x396d0] C [ntdll.dll+0x39620] C [imon.dll+0x446e] Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) J java.net.SocketInputStream.socketRead0(Ljava/io/FileDescriptor;[BIII)I J java.net.SocketInputStream.read([BII)I J com.sun.jini.jeri.internal.mux.StreamConnectionIO$1.read(Ljava/nio/ByteBuffer;)I J com.sun.jini.jeri.internal.mux.StreamConnectionIO$Reader.run()V J com.sun.jini.thread.ThreadPool$Worker.run()V j java.lang.Thread.run()V+11 v ~StubRoutines::call_stub Jasmina Boitcheva [e-mail] Submitted On 07-JAN-2008 rhastings1 We are seeing this as well. What is the new bug number for tracking this? # # An unexpected error has been detected by Java Runtime Environment: # # EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x7c82be3e, pid=3884, tid=4008 # # Java VM: Java HotSpot(TM) Client VM (1.6.0_03-b05 mixed mode) # Problematic frame: # C [ntdll.dll+0x2be3e] # # If you would like to submit a bug report, please visit: # http://java.sun.com/webapps/bugreport/crash.jsp # --------------- T H R E A D --------------- Current thread (0x26980400): JavaThread "http-8080-Processor22" daemon [_thread_in_native, id=4008] siginfo: ExceptionCode=0xc0000005, reading address 0x2054453f Registers: EAX=0x00000380, EBX=0x00000401, ECX=0x0000037f, EDX=0x20544547 ESP=0x285decb0, EBP=0x285decbc, ESI=0x005d0178, EDI=0x272f0000 EIP=0x7c82be3e, EFLAGS=0x00010283 Top of Stack: (sp=0x285decb0) 0x285decb0: 005d0000 00000482 26dc1000 285dece0 0x285decc0: 7c82b561 005d0178 00000482 26dc1000 0x285decd0: 005d0000 26be0000 26be0000 022f0000 0x285dece0: 285ded28 7c82ba81 00000482 00dc1000 0x285decf0: 00000482 005d0000 26dc1000 00000000 0x285ded00: 00000410 26dc3000 285ded24 26be0000 0x285ded10: 00000000 00000000 26dc0ff0 00000000 0x285ded20: 00000000 00000082 285dee0c 7c82b646 Instructions: (pc=0x7c82be3e) 0x7c82be2e: 57 64 8b 14 8a 83 c2 08 3b f2 0f 84 d5 fe ff ff 0x7c82be3e: 0f b7 5a f8 3b 5d 0c 73 0b 8b 12 3b f2 75 f1 e9 Stack: [0x285a0000,0x285e0000), sp=0x285decb0, free space=251k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) C [ntdll.dll+0x2be3e] C [ntdll.dll+0x2b561] C [ntdll.dll+0x2ba81] C [ntdll.dll+0x2b646] C [MSVCR71.dll+0x218a] C [net.dll+0x7129] J java.net.SocketInputStream.socketRead0(Ljava/io/FileDescriptor;[BIII)I J java.net.SocketInputStream.read([BII)I J org.apache.coyote.http11.InternalInputBuffer.fill()Z J org.apache.coyote.http11.InternalInputBuffer.parseRequestLine()V J org.apache.coyote.http11.Http11Processor.process(Ljava/io/InputStream;Ljava/io/OutputStream;)V j org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Lorg/apache/tomcat/util/net/TcpConnection;[Ljava/lang/Object;)V+113 j org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(Ljava/net/Socket;Lorg/apache/tomcat/util/net/TcpConnection;[Ljava/lang/Object;)V+45 j org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt([Ljava/lang/Object;)V+102 j org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run()V+167 j java.lang.Thread.run()V+11 v ~StubRoutines::call_stub ... VM Arguments: jvm_args: -Dcatalina.home=d:\Program Files\Apache Software Foundation\Tomcat 5.5 -Dcatalina.base=d:\Program Files\Apache Software Foundation\Tomcat 5.5 -Djava.endorsed.dirs=d:\Program Files\Apache Software Foundation\Tomcat 5.5\common\endorsed -Djava.io.tmpdir=d:\Program Files\Apache Software Foundation\Tomcat 5.5\temp -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.util.logging.config.file=d:\Program Files\Apache Software Foundation\Tomcat 5.5\conf\logging.properties vfprintf -Xmx512m java_command: <unknown> Launcher Type: generic Environment Variables: CLASSPATH=.;D:\Program Files\Java\jre1.6.0_03\lib\ext\QTJava.zip PATH=C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program Files\Intel\DMIX;D:\Program Files\QuickTime\QTSystem\ OS=Windows_NT PROCESSOR_IDENTIFIER=x86 Family 6 Model 15 Stepping 6, GenuineIntel --------------- S Y S T E M --------------- OS: Windows Server 2003 family Build 3790 Service Pack 2 CPU:total 4 (2 cores per cpu, 1 threads per core) family 6 model 15 stepping 6, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3 Memory: 4k page, physical 2097151k(2097151k free), swap 4194303k(4194303k free) vm_info: Java HotSpot(TM) Client VM (1.6.0_03-b05) for windows-x86, built on Sep 24 2007 22:24:33 by "java_re" with unknown MS VC++:1310 Submitted On 01-FEB-2008 i encountered it and really want to know what to do to avoid it because my work is stopped as i was using Matlab Distributed Computing Engine but this error comes up and stops MDCE automatically :( Submitted On 09-APR-2008 mbondurant We are also seeing this using 1.6. Here is a dump from our case: # Java VM: Java HotSpot(TM) 64-Bit Server VM (1.6.0_01-b06 mixed mode) siginfo: ExceptionCode=0xc0000005, reading address 0xffffffffffffffff Registers: EAX=0x000000000000047f, EBX=0x0000000006080000, ECX=0x6c69662f20544537, EDX=0x6c69662f20544547 ESP=0x0000000035f0e9c0, EBP=0x0000000000002000, ESI=0x0000000000ac02c8, EDI=0x0000000041249000 EIP=0x0000000077efd689, EFLAGS=0x0000000000010202 Top of Stack: (sp=0x0000000035f0e9c0) 0x0000000035f0e9c0: 0000000000000001 0000000000002000 0x0000000035f0e9d0: 0000000000000000 0000000000000000 0x0000000035f0e9e0: 0000000000ac0000 0000000077ef540f 0x0000000035f0e9f0: 0000000008916c00 0000000000000000 0x0000000035f0ea00: 0000000041249000 0000000000ac0000 0x0000000035f0ea10: 00000000116dd645 0000000000000000 0x0000000035f0ea20: 0000000010d00001 0000000000002010 0x0000000035f0ea30: 0000000041249000 0000000000000000 0x0000000035f0ea40: 0000000035f0eaa8 000007ff771bdf6f 0x0000000035f0ea50: 00000000c000020d 0000000000d631a2 0x0000000035f0ea60: 0000000000000000 0000000008916d90 0x0000000035f0ea70: 0000000000000001 000007ff771b1c56 0x0000000035f0ea80: 0000000008916d90 00000000091f4550 0x0000000035f0ea90: 0000000041249010 0000000000000000 0x0000000035f0eaa0: 0000000035f0eb60 0000000000012017 0x0000000035f0eab0: 0000000000000000 0000000000000018 Instructions: (pc=0x0000000077efd689) 0x0000000077efd679: 10 48 8b 7c 24 48 48 8b 5c 24 40 48 83 c4 28 c3 0x0000000077efd689: 0f b7 41 08 41 3b c3 0f 85 8c 77 ff ff 4b 89 0c Stack: [0x0000000035e10000,0x0000000035f10000), sp=0x0000000035f0e9c0, free space=1018k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) C [ntdll.dll+0x3d689] [error occurred during error reporting, step 120, id 0xc0000005] Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) J java.net.SocketInputStream.socketRead0(Ljava/io/FileDescriptor;[BIII)I j java.net.SocketInputStream.read([BII)I+84 J org.apache.coyote.http11.InternalInputBuffer.parseRequestLine()V j org.apache.coyote.http11.Http11Processor.process(Ljava/io/InputStream;Ljava/io/OutputStream;)V+327 j org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Lorg/apache/tomcat/util/net/TcpConnection;[Ljava/lang/Object;)V+113 j org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(Ljava/net/Socket;Lorg/apache/tomcat/util/net/TcpConnection;[Ljava/lang/Object;)V+45 j org.apache.tomcat.util.net.MasterSlaveWorkerThread.run()V+32 j java.lang.Thread.run()V+11 v ~StubRoutines::call_stub PLEASE NOTE: JDK6 is formerly known as Project Mustang