Solved

Java - strange issue when Logger.global hangs

Posted on 2010-08-28
12
550 Views
Last Modified: 2012-06-21
Hi.

I'm using a standard Java logger (please see enclosed code). But application suddenly hangs! :-(

In particular - there is a "main" application which starts a copy of itself with different command line parameters. All instances of my Java application writes data to own log file (something like mainXXX.log and executorXXX.log, where XXX is a current timestamp).

I see that all test cases produce mainXXX.log =~ 1.3kb, executorXXX.log =~ 320-390 bytes size log files. And then it hang.

I may think the main application could hang but a child application cannot hang for sure!
Because it is very simple and straight-forward, and also I have noticed that if I comment out some log write calls then application will step a bit more further and hang in the different place than before, which I think means that it hanging exactly because of logging.

For example - I see that a child java.exe process hang after 5 writes to a log file. And that could be any log writes. By commenting log writes at the beginning of application I can see it steps futher.
If I kill the main java.exe process the child process unfreeze and continue working and logging.

Do you have any ideas - why it could hang in such way?
Could it be a blocking I/O calls for a java process?
Any ideas - what to check? How to solve?

Could it be because main application writes some data to STDIN for a child java process and then waiting data from STDOUT?

If standard logger is "bad" can you then recommend some good logger solution for Java?

Regards,
Dmitry.
public class MiscUtils

{

	public static void toLogger(Level pLevel, String pMsg)

	{

		if (!Logger.global.isLoggable(pLevel)) return ;

		

		Logger.global.log(pLevel, pMsg);

		if (consoleOutputEnabled) 

			System.out.println(pMsg);

	}

}



// usage example 1

MiscUtils.toLogger(Level.INFO, "* Open db connection...");



// usage example 2

MiscUtils.toLogger(Level.WARNING, String.format("! Trying to load db driver (%1$s)...", pDrvName));

Open in new window

0
Comment
Question by:Dmitry_Bond
  • 4
  • 3
  • 2
  • +2
12 Comments
 
LVL 10

Accepted Solution

by:
gordon_vt02 earned 250 total points
ID: 33553387
How are you starting the child applications?  If you are using Runtime.exec(), make sure you are reading both the stdout and stderr streams of the child processes.  Those are buffered and if the buffer gets full, the process will hang.  I think the same thing applies if you are using ProcessBuilder.
0
 
LVL 10

Expert Comment

by:gordon_vt02
ID: 33553394
If that doesn't solve your problem, I'd run the code in a debugging session to see where the main app is actually locking up.
0
 
LVL 92

Assisted Solution

by:objects
objects earned 250 total points
ID: 33555434
take a thread dump to see why it is hanging

> If standard logger is "bad" can you then recommend some good logger solution for Java?

try log4j or slf4j
0
 

Author Comment

by:Dmitry_Bond
ID: 33556390
Yes, I'm using Runtime.exec() to run another copy of java.exe.
From created process I'm reading/writing StdIn and StoOut only, but no StdErr.
I assume it should not be the issue. Example - who cares if child process writing something to StdErr, if StdErr is not read by some other processes no issues expected. Correct?

I'm new in Java tools, could you please provide some details - how to do "thread dump"?

PS. I'm using "Eclipse Java EE IDE for Web Developers. Version: Helios Release. Build id: 20100617-1415".
0
 
LVL 92

Expert Comment

by:objects
ID: 33556420
> who cares if child process writing something to StdErr, if StdErr is not read by some other processes no issues expected. Correct?

incorrect

http://helpdesk.objects.com.au/java/runtime-exec-locks-up
0
Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

 

Author Comment

by:Dmitry_Bond
ID: 33556512
I think your example is not describing what we are discussing. Because in the example you provided it use only STDIN, STDOUT and there is nothing about STDERR. So, or example is wrong or idea shown incorrectly. Choose one option please.

Next thing - absolutely for sure the child process I'm creating in application is not writing anything to STDERR because it is quite simple and all executing I can see in a log - there are no any hints telling about errors or such stuff. So, I think that is not a "stream overflow and hang"  issue.
Also because - it write/read only few data to STDIN/STDOUT - about 300-500 chars before it hang, I doubt buffer size is so small... how could I know the default buffer size?


Update:
I just replaced logging with log4j (1.2.16) - everything starts to working fine - no more hangs, no more problems.
Could it be because of bugs in standard Logger object?
If yes, then rhetoric question - why Sun/Oracle/IBM included such "bad" Logger object into a standard?!

PS. I'm running it using JRE 1.5 - "IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Windows Vista x86-32 j9vmwi3223-20070426 (JIT enabled)". Seems exactly the Java 1.5 have be the target for project.

Just for a case I just also tested the same with Java 1.6.0_21 - it also hangs. :-\
0
 
LVL 92

Expert Comment

by:objects
ID: 33557085
> Because in the example you provided it use only STDIN, STDOUT and there is nothing about STDERR. So, or example is wrong or idea shown incorrectly.

STDERR is the same as STDOUT
point is you need to consume all output from the process
0
 

Author Comment

by:Dmitry_Bond
ID: 33557124
> STDERR is the same as STDOUT. point is you need to consume all output from the process

Ok. Then it cannot be the reason of mentioned issue. Because the main process reads STDOUT.
0
 
LVL 7

Expert Comment

by:rumi78
ID: 33581637
Dear Dmitry_Bond,

Both of mentioned propositions may cause it:
1. Blocking IO on logging - make sure you are using separate log files on fist and second jvm, on windows you can not even remove file opened for reading :D, even so called Administrator rights are not sufficient.
2. STDOUT and STDERR are separate pipes, not reading any of them may cause hang, do not use e.printStackTrace(), you can simply replace it with e.printStackTrace(System.out), remember to surround the whole invocation with try/catch(Exception e){e.printStackTrace(System.out);}, Most probably some RuntimeException (like NullPointerException) is causing writing to STDERR.

rgds
rumi
0
 

Author Comment

by:Dmitry_Bond
ID: 33609384
Absolutely for sure (without even a slight doubt!):
1)  - there are separate log files for different jvm. Actually only 2 jvm - every writing own log file.
2) - STDERR is not written at all! And that is true - no any errors appears while application working.

Also, as I already specified - I had only replaced standard Java logger with log4j and all things starts to working fine. Which means (as I see it) = STANDARD LOGGER HAS BUGS making application hang.

Any comments? Any ideas how to check what application is doing at the moment?
Stacktrace, etc? Can you recommend any tools to debug/trace/check java applications?
0
 
LVL 26

Expert Comment

by:mrcoffee365
ID: 37126730
This question has been classified as abandoned and is closed as part of the Cleanup Program. See the recommendation for more details.
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

Suggested Solutions

Title # Comments Views Activity
noX challenge 17 77
topping2 challenge 13 61
Java and GPO 11 47
eclipse formatting 6 43
By the end of 1980s, object oriented programming using languages like C++, Simula69 and ObjectPascal gained momentum. It looked like programmers finally found the perfect language. C++ successfully combined the object oriented principles of Simula w…
Java Flight Recorder and Java Mission Control together create a complete tool chain to continuously collect low level and detailed runtime information enabling after-the-fact incident analysis. Java Flight Recorder is a profiling and event collectio…
Viewers learn about the “while” loop and how to utilize it correctly in Java. Additionally, viewers begin exploring how to include conditional statements within a while loop and avoid an endless loop. Define While Loop: Basic Example: Explanatio…
This tutorial explains how to use the VisualVM tool for the Java platform application. This video goes into detail on the Threads, Sampler, and Profiler tabs.

760 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now