Solved

Thread Problem

Posted on 2002-04-07
17
511 Views
Last Modified: 2012-06-27
I wrote a program that downloads a list of urls using a large number of threads to speed up the process. However, at random points in the downloading process, the JVM breaks, saying there was a 'problematic thread' at some memory address.  

My basic question: Does anyone know of any general types of problems using threads that cause the JVM to break frequently?

The only sychronization issues that I have are that every thread passes a string to a synchronized static method, which writes out the string to a RandomAccessFile.  I don't know if this would be causing any problems, but it's a rather simple thing, so I wouldn't think so.  

Has anyone run into similar problems on their systems?

I'm running on Windows 2000 with jdk 1.3.1_02.

Cheers,
Brett
0
Comment
Question by:brgordon
  • 6
  • 3
  • 2
  • +5
17 Comments
 
LVL 3

Expert Comment

by:rjackman
ID: 6924589
can u post the error (message obtained by using printStackTrace() method)
cheers
RJackman
0
 

Author Comment

by:brgordon
ID: 6924615
rjackman,

Sometimes there is a stacktrace, but in general, the JVM just crashes.

One error looks about like this:

#Java version: Java HotSpot(TM) Client VM
#
#HotSpot Virtual Machine Error, Internal Error
#
#System received a SEGMENTATION_FAULT_EXCEPTION at [hex address]
#Please report this error to the Java Bug Database
#Error ID: [some very long number]
#
#Problematic thread: prio=5 tid=[some hex address] nid=29 lwp_id=42990 runnable
#

Otherwise, the errors look like:

#An unexpected exception has been detected in native code outside the VM
#Unexpected signal: SEGMENTATION_FAULT_EXCEPTION
#Function Name: unknown
#Library: unknown
#
#[begin very long trace through all the native libraries]
#
#

My application uses a large number of threads, some which remain for a long time, while others are created and destroyed quickly.  The system seems to be more stable when it runs with fewer threads.  Are there some hidden parameters that might be too low?  

If you need anymore info, let me know.

Thanks,
Brett
0
 
LVL 92

Expert Comment

by:objects
ID: 6924665
You using JNI?
0
Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

 
LVL 3

Expert Comment

by:exorcist
ID: 6924814
You might want to try another JVM vendor. Or another version.

Calling a synchronized method from multiple threads is perfectly fine and shouldnt cause any errors.

0
 
LVL 16

Expert Comment

by:heyhey_
ID: 6925094
> You might want to try another JVM vendor. Or another version.

or you can try disabling the JIT engine (java -classic MainlClass)
0
 
LVL 5

Expert Comment

by:LexZEUS
ID: 6925254
I have some problem... but mine is using java.io.Socket, my suggestion is not to create too many active URL retriever. Maybe you need to use ThreadGroup to see current active threads, if there are 20/more active threads you may need to pause incoming url queues.
I manage to solve my problem with trial & error, adjusting/reducing from 30+ active threads until it is stable at 20 threads... Win98 Intel P3700MhZ with 128MB RAM
0
 
LVL 5

Expert Comment

by:LexZEUS
ID: 6925257
my first words above:
I have some problem
supposed to be
I have "same" problem

pardon my typo.. :)
0
 
LVL 3

Expert Comment

by:exorcist
ID: 6925268
Try to deploy your application on a Linux system and I bet you won't get this error anymore.

As far as I can remember, I had a similar error and switched to run this program on Linux only.

It's worth a try.
0
 

Author Comment

by:brgordon
ID: 6925449
Response to all:

exorcist,
1.  I have tried both jdk1.3.1_02 and jdk1.4, using both the client and server HotSpot VMs.
2.  I have switched it too linux (RH 7.2) and to Unix.  Same problems.

objects,
No, I'm not using JNI.

LexZEUS,
The architecture is such that a single downloading thread controls multiple downloading other downloading threads, which in turn, create another thread to monitor the downloading process (in case it times out).  I have been able to run it for a reasonable amount of time having 10 main downloading threads and 10 secondary downloading threads (achieving a retrieval rate of ~30 pages/second).  Thus, there could be about 150 to 200 threads alive at any given point in time.
I'm assuming that at a base level URLConnection objects use the Socket class, so perhaps I'm having a similar problem.  In terms of reducing the number of active threads, what exact approach did you take?  Did you switch between ThreadGroups, each with 20 threads, or was your total number of threads 20?


Thanks all,

Brett
0
 
LVL 16

Expert Comment

by:heyhey_
ID: 6925466
Comment

From: heyhey_
Date: 04/08/2002 05:15AM PST

> You might want to try another JVM vendor. Or another version.

or you can try disabling the JIT engine (java -classic MainlClass)
0
 

Author Comment

by:brgordon
ID: 6925486
heyhey,

woops.....

I'll try that, too.

Brett
0
 

Author Comment

by:brgordon
ID: 6926444
heyhey,

Why do you think getting rid of JIT will help?
I just started it going using that, so we'll see in a little while.

Everyone,

I just cleaned out my system of all java related things, and did a complete reinstall of j2sdk1.4.0.  

On the previous run, the program quickly broke.  Windows  shut the process down.  Java printed the following:


Unexpected Signal : EXCEPTION_ACCESS_VIOLATION occurred at PC=0x6D3B6F55
Function=JVM_RegisterUnsafeMethods+0x11DF
Library=c:\j2sdk1.4.0\jre\bin\client\jvm.dll

Current Java thread:
        at java.lang.Throwable.fillInStackTrace(Native Method)
        - locked <02A42D20> (a java.lang.NullPointerException)
        at java.lang.Throwable.<init>(Throwable.java:180)
        at java.lang.Exception.<init>(Exception.java:29)
        at java.lang.RuntimeException.<init>(RuntimeException.java:32)
        at java.lang.NullPointerException.<init>(NullPointerException.java:36)
[error occured during error reporting]


That's it.  I'm guessing it couldn't finish reporting because Windows cut the process off, but that's just a guess.

Brett
0
 
LVL 4

Expert Comment

by:pellep
ID: 6926561
just a question. You mentioned you were using a monitoring thread to check if the worker-threads are still alive. Are you anywhere executing the stop() (Thread.stop()) function on your worker threads?  
This line in your error sorta caught my attention
Function=JVM_RegisterUnsafeMethods+0x11DF
If you are, you should find a different method of stopping your dead threads. There is some mention in the JDK about the stop() function beeing 'unsafe'.

The whole SEGMENTATION_FAULT_EXCEPTION thing looks kinda 'fishy'. Segmentation faults will turn up for instance if you in C++ try to do
delete(this);
ie a class deleting itself. I have also seen it turn up when dooing multithreaded programing in which one thread deletes/destroys/de-allocates an object/memory-area used by another thread.
My little theory on this (subject to scrutiny from other experts):
The problem might lie with the socket. A socket, basically, could be seen as a memory area representing the actual buffer for received bytes. You have two threads operating on this memory-area: the application-thread waiting for data to enter the buffer and the underlying IP-stack waiting for data from the network to put on the buffer. On the side is a semaphore (monitor in java-speak) that the IP-stack thread signals and the application-thread waits on.
Now, I can see two scenarious where this might turn nasty
- The worker-thread in your application gets stopped using the Thread.stop() function. This is an inherently dangerous method, and the JDK mentions that monitors used by the thread beeing stopped may get screwed up and other . It's conceivable that you in this scenario may end up with the socket-semaphore beeing in an 'inconsistent state' and therefor causing the segmentation fault when the underlying socket implementation tries to signal it.
- Statistics and bad luck/bad JVM implementation. Since you have so many threads with their own sockets beeing created and destroyed there might be a possibility that the garbage-collector simply can't 'keep up'. What could happen then is that between the time the garbage-collector/socket subsystem starts de-allocating memory for derefrenced classes, thereby releasing the socket buffer memory-area, and the actual closing of the socket handle, the underlying IP-stack thread may try to write to the de-allocated socket buffer.

As you can hear, there are a lot of 'if', 'maybe', 'could', 'may' and 'might' in here and that's because I'm guessing a lot. You could post a reference to this question in the C++ area, since it's a strong possibility this is an underlying problem.

PAP
0
 

Author Comment

by:brgordon
ID: 6926662
heyhey,

I turned off the JIT engine, but it still crashed after a little while (running many threads).

Brett
0
 
LVL 1

Expert Comment

by:Moondancer
ID: 6958038
Is more needed here?
Moondancer - EE Moderator
0
 

Author Comment

by:brgordon
ID: 6958529
All,

I have tested my code on another Windows machine, and it works there, but not on my main machine.  I traced the exception down lower, and the problem actually appears to be in java.io.Socket.read().

Anyone found wierd problems with reading from sockets?

Brett

0
 
LVL 16

Accepted Solution

by:
heyhey_ earned 200 total points
ID: 6959080
I have seen strange socket behaviour twice. (forst one - application on NT + IBM JDK 1.1.7, second one - applet with JRE 1.4.x)

in both cases when the system works on a high load (a lot of traffic over a lot of connections) some of the bytes sent over TCP socket were never received on the other end. (let's say that sender sends three 4-byte headers and  receiver receives only first and third).

both situation were pretty annonying, very hard to find out the problem (because I trust the TCP stack) and impossible to fix.



in your situation there are probably problems with OS network stack or some nasty driver problems.
0

Featured Post

Free Tool: Subnet Calculator

The subnet calculator helps you design networks by taking an IP address and network mask and returning information such as network, broadcast address, and host range.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Introduction Java can be integrated with native programs using an interface called JNI(Java Native Interface). Native programs are programs which can directly run on the processor. JNI is simply a naming and calling convention so that the JVM (Java…
Introduction This article is the last of three articles that explain why and how the Experts Exchange QA Team does test automation for our web site. This article covers our test design approach and then goes through a simple test case example, how …
Viewers will learn about if statements in Java and their use The if statement: The condition required to create an if statement: Variations of if statements: An example using if statements:
Viewers will learn how to properly install Eclipse with the necessary JDK, and will take a look at an introductory Java program. Download Eclipse installation zip file: Extract files from zip file: Download and install JDK 8: Open Eclipse and …

790 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question