Solved

Thread Problem

Posted on 2002-04-07
17
507 Views
Last Modified: 2012-06-27
I wrote a program that downloads a list of urls using a large number of threads to speed up the process. However, at random points in the downloading process, the JVM breaks, saying there was a 'problematic thread' at some memory address.  

My basic question: Does anyone know of any general types of problems using threads that cause the JVM to break frequently?

The only sychronization issues that I have are that every thread passes a string to a synchronized static method, which writes out the string to a RandomAccessFile.  I don't know if this would be causing any problems, but it's a rather simple thing, so I wouldn't think so.  

Has anyone run into similar problems on their systems?

I'm running on Windows 2000 with jdk 1.3.1_02.

Cheers,
Brett
0
Comment
Question by:brgordon
  • 6
  • 3
  • 2
  • +5
17 Comments
 
LVL 3

Expert Comment

by:rjackman
ID: 6924589
can u post the error (message obtained by using printStackTrace() method)
cheers
RJackman
0
 

Author Comment

by:brgordon
ID: 6924615
rjackman,

Sometimes there is a stacktrace, but in general, the JVM just crashes.

One error looks about like this:

#Java version: Java HotSpot(TM) Client VM
#
#HotSpot Virtual Machine Error, Internal Error
#
#System received a SEGMENTATION_FAULT_EXCEPTION at [hex address]
#Please report this error to the Java Bug Database
#Error ID: [some very long number]
#
#Problematic thread: prio=5 tid=[some hex address] nid=29 lwp_id=42990 runnable
#

Otherwise, the errors look like:

#An unexpected exception has been detected in native code outside the VM
#Unexpected signal: SEGMENTATION_FAULT_EXCEPTION
#Function Name: unknown
#Library: unknown
#
#[begin very long trace through all the native libraries]
#
#

My application uses a large number of threads, some which remain for a long time, while others are created and destroyed quickly.  The system seems to be more stable when it runs with fewer threads.  Are there some hidden parameters that might be too low?  

If you need anymore info, let me know.

Thanks,
Brett
0
 
LVL 92

Expert Comment

by:objects
ID: 6924665
You using JNI?
0
 
LVL 3

Expert Comment

by:exorcist
ID: 6924814
You might want to try another JVM vendor. Or another version.

Calling a synchronized method from multiple threads is perfectly fine and shouldnt cause any errors.

0
 
LVL 16

Expert Comment

by:heyhey_
ID: 6925094
> You might want to try another JVM vendor. Or another version.

or you can try disabling the JIT engine (java -classic MainlClass)
0
 
LVL 5

Expert Comment

by:LexZEUS
ID: 6925254
I have some problem... but mine is using java.io.Socket, my suggestion is not to create too many active URL retriever. Maybe you need to use ThreadGroup to see current active threads, if there are 20/more active threads you may need to pause incoming url queues.
I manage to solve my problem with trial & error, adjusting/reducing from 30+ active threads until it is stable at 20 threads... Win98 Intel P3700MhZ with 128MB RAM
0
 
LVL 5

Expert Comment

by:LexZEUS
ID: 6925257
my first words above:
I have some problem
supposed to be
I have "same" problem

pardon my typo.. :)
0
 
LVL 3

Expert Comment

by:exorcist
ID: 6925268
Try to deploy your application on a Linux system and I bet you won't get this error anymore.

As far as I can remember, I had a similar error and switched to run this program on Linux only.

It's worth a try.
0
Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

 

Author Comment

by:brgordon
ID: 6925449
Response to all:

exorcist,
1.  I have tried both jdk1.3.1_02 and jdk1.4, using both the client and server HotSpot VMs.
2.  I have switched it too linux (RH 7.2) and to Unix.  Same problems.

objects,
No, I'm not using JNI.

LexZEUS,
The architecture is such that a single downloading thread controls multiple downloading other downloading threads, which in turn, create another thread to monitor the downloading process (in case it times out).  I have been able to run it for a reasonable amount of time having 10 main downloading threads and 10 secondary downloading threads (achieving a retrieval rate of ~30 pages/second).  Thus, there could be about 150 to 200 threads alive at any given point in time.
I'm assuming that at a base level URLConnection objects use the Socket class, so perhaps I'm having a similar problem.  In terms of reducing the number of active threads, what exact approach did you take?  Did you switch between ThreadGroups, each with 20 threads, or was your total number of threads 20?


Thanks all,

Brett
0
 
LVL 16

Expert Comment

by:heyhey_
ID: 6925466
Comment

From: heyhey_
Date: 04/08/2002 05:15AM PST

> You might want to try another JVM vendor. Or another version.

or you can try disabling the JIT engine (java -classic MainlClass)
0
 

Author Comment

by:brgordon
ID: 6925486
heyhey,

woops.....

I'll try that, too.

Brett
0
 

Author Comment

by:brgordon
ID: 6926444
heyhey,

Why do you think getting rid of JIT will help?
I just started it going using that, so we'll see in a little while.

Everyone,

I just cleaned out my system of all java related things, and did a complete reinstall of j2sdk1.4.0.  

On the previous run, the program quickly broke.  Windows  shut the process down.  Java printed the following:


Unexpected Signal : EXCEPTION_ACCESS_VIOLATION occurred at PC=0x6D3B6F55
Function=JVM_RegisterUnsafeMethods+0x11DF
Library=c:\j2sdk1.4.0\jre\bin\client\jvm.dll

Current Java thread:
        at java.lang.Throwable.fillInStackTrace(Native Method)
        - locked <02A42D20> (a java.lang.NullPointerException)
        at java.lang.Throwable.<init>(Throwable.java:180)
        at java.lang.Exception.<init>(Exception.java:29)
        at java.lang.RuntimeException.<init>(RuntimeException.java:32)
        at java.lang.NullPointerException.<init>(NullPointerException.java:36)
[error occured during error reporting]


That's it.  I'm guessing it couldn't finish reporting because Windows cut the process off, but that's just a guess.

Brett
0
 
LVL 4

Expert Comment

by:pellep
ID: 6926561
just a question. You mentioned you were using a monitoring thread to check if the worker-threads are still alive. Are you anywhere executing the stop() (Thread.stop()) function on your worker threads?  
This line in your error sorta caught my attention
Function=JVM_RegisterUnsafeMethods+0x11DF
If you are, you should find a different method of stopping your dead threads. There is some mention in the JDK about the stop() function beeing 'unsafe'.

The whole SEGMENTATION_FAULT_EXCEPTION thing looks kinda 'fishy'. Segmentation faults will turn up for instance if you in C++ try to do
delete(this);
ie a class deleting itself. I have also seen it turn up when dooing multithreaded programing in which one thread deletes/destroys/de-allocates an object/memory-area used by another thread.
My little theory on this (subject to scrutiny from other experts):
The problem might lie with the socket. A socket, basically, could be seen as a memory area representing the actual buffer for received bytes. You have two threads operating on this memory-area: the application-thread waiting for data to enter the buffer and the underlying IP-stack waiting for data from the network to put on the buffer. On the side is a semaphore (monitor in java-speak) that the IP-stack thread signals and the application-thread waits on.
Now, I can see two scenarious where this might turn nasty
- The worker-thread in your application gets stopped using the Thread.stop() function. This is an inherently dangerous method, and the JDK mentions that monitors used by the thread beeing stopped may get screwed up and other . It's conceivable that you in this scenario may end up with the socket-semaphore beeing in an 'inconsistent state' and therefor causing the segmentation fault when the underlying socket implementation tries to signal it.
- Statistics and bad luck/bad JVM implementation. Since you have so many threads with their own sockets beeing created and destroyed there might be a possibility that the garbage-collector simply can't 'keep up'. What could happen then is that between the time the garbage-collector/socket subsystem starts de-allocating memory for derefrenced classes, thereby releasing the socket buffer memory-area, and the actual closing of the socket handle, the underlying IP-stack thread may try to write to the de-allocated socket buffer.

As you can hear, there are a lot of 'if', 'maybe', 'could', 'may' and 'might' in here and that's because I'm guessing a lot. You could post a reference to this question in the C++ area, since it's a strong possibility this is an underlying problem.

PAP
0
 

Author Comment

by:brgordon
ID: 6926662
heyhey,

I turned off the JIT engine, but it still crashed after a little while (running many threads).

Brett
0
 
LVL 1

Expert Comment

by:Moondancer
ID: 6958038
Is more needed here?
Moondancer - EE Moderator
0
 

Author Comment

by:brgordon
ID: 6958529
All,

I have tested my code on another Windows machine, and it works there, but not on my main machine.  I traced the exception down lower, and the problem actually appears to be in java.io.Socket.read().

Anyone found wierd problems with reading from sockets?

Brett

0
 
LVL 16

Accepted Solution

by:
heyhey_ earned 200 total points
ID: 6959080
I have seen strange socket behaviour twice. (forst one - application on NT + IBM JDK 1.1.7, second one - applet with JRE 1.4.x)

in both cases when the system works on a high load (a lot of traffic over a lot of connections) some of the bytes sent over TCP socket were never received on the other end. (let's say that sender sends three 4-byte headers and  receiver receives only first and third).

both situation were pretty annonying, very hard to find out the problem (because I trust the TCP stack) and impossible to fix.



in your situation there are probably problems with OS network stack or some nasty driver problems.
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

An old method to applying the Singleton pattern in your Java code is to check if a static instance, defined in the same class that needs to be instantiated once and only once, is null and then create a new instance; otherwise, the pre-existing insta…
Java Flight Recorder and Java Mission Control together create a complete tool chain to continuously collect low level and detailed runtime information enabling after-the-fact incident analysis. Java Flight Recorder is a profiling and event collectio…
Viewers learn how to read error messages and identify possible mistakes that could cause hours of frustration. Coding is as much about debugging your code as it is about writing it. Define Error Message: Line Numbers: Type of Error: Break Down…
Viewers will learn about arithmetic and Boolean expressions in Java and the logical operators used to create Boolean expressions. We will cover the symbols used for arithmetic expressions and define each logical operator and how to use them in Boole…

759 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now