Learn how to a build a cloud-first strategyRegister Now

x
?
Solved

Java App Hangs as it downloads.....

Posted on 2006-04-08
28
Medium Priority
?
385 Views
Last Modified: 2010-08-05
I did not program this and I have no java programming knowledge so just looking for some thoughts on what could be wrong here... I personally think it has something to do with a buffer or some sort getting overloaded. Because when it does 500 little files it does them in like 3 minutes but when it does one huge file it hangs and takes like 2 hour to download 1mb.

Ok I have a java app that downloads xml files from a server. As the Java app downloads these files it diplays the information on the screen in a command prompt. if these files are 1 - 50k they seem to fly and take just about 1 second to download. If the file is 300k it can take 3 hours to download. I could post the code but not sure if the code is the issue or is it some setting in the java environment I need to set. Can someone help me figure this out.

Another note... the java app runs on a Windows XP SP2 workstation and use command prompt to run.

J
0
Comment
Question by:jsuttor
  • 13
  • 9
  • 6
28 Comments
 
LVL 86

Expert Comment

by:CEHJ
ID: 16409597
You maybe right about it being buffer-related. Need to see the code though
0
 
LVL 3

Author Comment

by:jsuttor
ID: 16409631
OK here are the .java file that runs from the batch file that executes it. Let me know if you need anything else.

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.PrintWriter;
import java.net.ServerSocket;
import java.net.Socket;

public class SocketReader
{
     ServerSocket server = null;
     Socket client = null;
     BufferedReader in = null;
     PrintWriter out = null;
     String line;
    String strTotal="";

     public void listenSocket()
     {
          try {
               server = new ServerSocket(9600);
          }
          catch (IOException e) {
               System.out.println("Could not listen on port 9600");
               System.exit(-1);
          }

          try {
               client = server.accept();
          }
          catch (IOException e) {
               System.out.println("Accept failed: 9600");
               System.exit(-1);
          }

          try {
               in = new BufferedReader(new InputStreamReader(client.getInputStream()));
               out = new PrintWriter(client.getOutputStream(), true); //Getting client's output stream coz its up for grabs
          }
          catch (IOException e) {
               System.out.println("Accept failed: 9600");
               System.exit(-1);
          }
          int iCount=0;
          int iChk=0;
          try {
               while ((line = in.readLine()) != null) {                                    
                     System.out.println(line);      
                     
                     if (line != null)
                         {
                            iCount=line.length();
                              if (iCount==3){ //(line.startsWith("<?xml")){
                                if (iChk!=0){    
                                    //XMLWriter.writeXML(strTotal.substring(5, strTotal.length()).trim());  
                                    strTotal="";
                                }      
                                    iChk=1;
                              }
                                strTotal=strTotal+"\n"+line;                              
                         }          
                     out.println(line);
                     out.flush();
                 
                    //XMLWriter.writeXML(line);
                    //Send data back to client. Not required. Just writing back to the client coz u have his output stream!
                    //out.println("Received string" + line);
               }

          }
          catch (IOException e) {
              //e.printStackTrace();
               System.out.println("Read failed");
               System.exit(-1);
         }
     }

     protected void finalize()
     {
          //Clean up
          try {
               in.close();
               out.close();
               server.close();
          }
          catch (IOException e) {
               System.out.println("Could not close.");
               System.exit(-1);
          }
     }

     public static void main(String[] args)
     {
          SocketReader frame = new SocketReader();
          frame.listenSocket();
     }
}
0
 
LVL 92

Expert Comment

by:objects
ID: 16409661
code looks fine, problem may be on the sending side.
0
VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

 
LVL 3

Author Comment

by:jsuttor
ID: 16410432
I would find it hard to believe that the problem would be in the sending side since it comes from ESPN and they have over 2000 people using the sending side and I am the only one that seems to have a problem with big files. Only thing I can think of is some sort of buffer problem with Windows or some sort of buffer size problem with the java programming. I feel it probably has something to do more with the buffer size for JAVA as when the data hangs JAVA's processor usage goes to 97+% and stays there until the data shows it is starting to come again.

J
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16410464
See if you get on better with this:


final int BUFFER_SIZE = 1 << 9; // 500 KiB buffer
StringBuilder strTotal = new StringBuilder(BUFFER_SIZE);
try {
               while ((line = in.readLine()) != null) {                                    
                     System.out.println(line);      
                     
                     if (line != null)
                         {
                            iCount=line.length();
                              if (iCount==3){ //(line.startsWith("<?xml")){
                                if (iChk!=0){    
                                    //XMLWriter.writeXML(strTotal.substring(5, strTotal.length()).trim());  
                                    strTotal.setLength(0);
                                    strTotal.ensureCapacity(BUFFER_SIZE);
                                }      
                                    iChk=1;
                              }
                                strTotal.append("\n").append(line);
                         }          
                     out.println(line);
                     out.flush();
                 

Not sure if this business

>>if (iCount==3){ //(line.startsWith("<?xml")){

is quite right though. Looks like you need something more like

if ((line = line.trim()).startsWith("<?xml")) {
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16410467
Typo:

>>final int BUFFER_SIZE = 1 << 9; // 500 KiB buffer

should be

final int BUFFER_SIZE = 1 << 19; // 500 KiB buffer
0
 
LVL 3

Author Comment

by:jsuttor
ID: 16411895
OK I hate to sound like the total moron but because I have no knowledge of java programming... Do I put that in the .class file and then complie it? or how does that work.... I am sorry I am asking lame questions but I don't know anything about JAVA programming. I strickly usually do web programming in ASP.

J
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16412338
>>Do I put that in the .class file and then complie it?

You put it in the .java file and compile it
0
 
LVL 92

Expert Comment

by:objects
ID: 16413132
first remove all the code (following) that stores the returned result to isolate whether the problem is with saving it or whether it is with reading it.


                   if (line != null)
                         {
                            iCount=line.length();
                              if (iCount==3){ //(line.startsWith("<?xml")){
                                if (iChk!=0){    
                                    //XMLWriter.writeXML(strTotal.substring(5, strTotal.length()).trim());  
                                    strTotal="";
                                }      
                                    iChk=1;
                              }
                                strTotal=strTotal+"\n"+line;                              
                         }          
                     out.println(line);
                     out.flush();
0
 
LVL 3

Author Comment

by:jsuttor
ID: 16422920
objects.... I removed the code like you suggested and compiled the code again and ran it and HOLY it ran so fast it was not even funny... going to try CEHJ suggestion...
0
 
LVL 92

Expert Comment

by:objects
ID: 16422967
ok, so that tells you the problem is with how you are saving the response.
looks like the problem line is either:

                                strTotal=strTotal+"\n"+line;                              


StringBuilder will spped that up, though you never actually use strTotal. What do you plan to use it for.

or:

                   out.println(line);

Buffering the output stream may help here.
Though why are you echoing back the line, is that necessary?
0
 
LVL 3

Author Comment

by:jsuttor
ID: 16423007
objects... I dont know anything... sorry...

I am sure the echo is so I can see the input stream...  it helps so I am able to tell its running properly...

I tried the code example that CEHJ and it does not work... it still hangs in the same place it was hanging before... The files that I am trying to download are anywhere from 1k to 3mb and its anything over about 30k (I think that hangs a little) but the ones that give me the biggest problem are the big ones like the 500k and up... Especially the 1mb and up files...

J

0
 
LVL 3

Author Comment

by:jsuttor
ID: 16423014
another note... I can start and stop each file that is large and the part where is starts to hang is the exact same byte everytime...
0
 
LVL 3

Author Comment

by:jsuttor
ID: 16423055
here is the XMLWriter Code... maybe the problem is here...

import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;

public class XMLWriter
{
      private static String pathToDirectory = "./xmlfiles/"; //Whatever the name, the directory better exist before u run this!
      /*
      Avoiding complex XML parsing which can be done using xml4j.jar or the formidable Apache Digester framework
      coz in this case only the filename needs to be extracted from the xml input,
      which can be done using simple string parsing
      For now, assuming the <STORYNUMBER> tag to contain the filename. Change to whatever is necessary.
      */

      public static void writeXML(String xmlStr)
      {
        //if (xmlStr.length()>3){
 
                   String fileName = getFileName(xmlStr);

                  try {
                        FileOutputStream fos = new FileOutputStream(pathToDirectory + fileName);
                        fos.write(xmlStr.getBytes());
                        fos.flush();
                        fos.close();
                  }
                  catch (FileNotFoundException e) {
                        e.printStackTrace();
                  }
                  catch (IOException e) {
                        e.printStackTrace();  //To change body of catch statement use File | Settings | File Templates.
                  }
      //}
}

      private static String getFileName(String xmlStr)
      {
            
            int start = xmlStr.indexOf("<KEYWORD>") + 9;
            int end = xmlStr.indexOf("</KEYWORD>");

            if (start == 8 || end == -1)
            return "KeywordNotPresent";

            String fileName = xmlStr.substring(start, end) + ".xml";
            return fileName;
            
      }

}
0
 
LVL 92

Expert Comment

by:objects
ID: 16423062
> I am sure the echo is so I can see the input stream...  it helps so I am able to tell its running properly...

thats not echoing it the screen, its echoing it back to the server.
try commenting it out, or replacing it with:

System.out.println(line);

>    out.flush();

u can get rid of that to
0
 
LVL 3

Author Comment

by:jsuttor
ID: 16423134
Ok getting rid of those two lines...

out.println(line);
out.flush();

Make the program do some wired things... So I switched the out.println(line); to System.out.println(line); and deleted the out.flush(); and same wierd things...

meaning... that the wire parser (connection client) starts to read the stream... There is a wire parser java client that creates a connection to the server I receive the stream from. But the stream writing part still works the same way it did before.

So the problem seems to be here here in the write like you said...

 if (line != null)
                         {
                            iCount=line.length();
                              if (iCount==3){ //(line.startsWith("<?xml")){
                                if (iChk!=0){    
                                    //XMLWriter.writeXML(strTotal.substring(5, strTotal.length()).trim());  
                                    strTotal="";
                                }      
                                    iChk=1;
                              }
                                strTotal=strTotal+"\n"+line;                              
                         }          
                     out.println(line);
                     out.flush();
0
 
LVL 92

Expert Comment

by:objects
ID: 16423188
ok, try it with just this:

if (line != null)
{
 out.println(line);
}
0
 
LVL 3

Author Comment

by:jsuttor
ID: 16423297
I hope this is what you meant...

          try {
               while ((line = in.readLine()) != null) {                                    
                     System.out.println(line);      
                     
                     if (line != null)
                         {
                            iCount=line.length();
                              if (iCount==3){ //(line.startsWith("<?xml")){
                                if (iChk!=0){    
                                    //XMLWriter.writeXML(strTotal.substring(5, strTotal.length()).trim());  
                                    strTotal="";
                                }      
                                    iChk=1;
                              }
                                strTotal=strTotal+"\n"+line;                              
                         }          
                 if (line != null)
                 {
                   out.println(line);
                 }
                     out.flush();
 

If so, give me the same result as the other stuff.
0
 
LVL 92

Expert Comment

by:objects
ID: 16423326
actually I meant just:

             while ((line = in.readLine()) != null) {                                    
                     System.out.println(line);
                     out.println(line);
             }

some of your earlier statement have me confused:

1> here is the XMLWriter Code... maybe the problem is here...

from what u have posted above that doesn't even get called (it is commented out)

2> Ok getting rid of those two lines...
>out.println(line);
>out.flush();
>Make the program do some wired things...

But earlier when you removed that whole block it ran ok (and fast)
What exactly were the *weird* things
0
 
LVL 3

Author Comment

by:jsuttor
ID: 16423359
1> here is the XMLWriter Code... maybe the problem is here...

from what u have posted above that doesn't even get called (it is commented out)

Ok like I stated I do not know much about programming so disregard this one since it commented out.

Ok the weird things... It might be hard to explain but I will try my best...

Ok I will start by explaining... I run two batch files one runs the wire parser (another java program) which creates a connection to the server... the second batch file runs this java file SocketReader.java which is the file we have been working on. When I take out those lines of code... The wire parser acts like its reading the file... This is wierd because right now with those lines there all it showns is the fact that its running the program. It does not show anything on the screen.

The SocketReader.java does not act funny. its does the same thing it usually does.... pauses after so many bytes.
0
 
LVL 92

Expert Comment

by:objects
ID: 16423392
problem may be in the other app, can u post it
0
 
LVL 3

Author Comment

by:jsuttor
ID: 16423413
The other app is written by ESPN... they have 2,000 people using it and I seem to be the only one that hired a lame programmer to try to develop mine... Anyway, they did not include the .java file in the .jar file only the .class files so I dont have any code to give you.
0
 
LVL 92

Expert Comment

by:objects
ID: 16423512
What exactly is it you are trying to achive?
Currently it reads xml from the connection, and writes the same xml back on the connection. Whats the point of that.
Writing the xml back seems to be whats causing you problems, and it would seem unnecessary.
0
 
LVL 3

Author Comment

by:jsuttor
ID: 16423549
I do not care if it writes back to the server. Who cares as long as the files end up on my system. So that means

What I am trying to achive is just to have the stream create the files in a timely manner. Right now it takes like 8 hours to download and write a 3 mb file... OUCH its killing me because it hold up all the other stuff. So basically just to capture the stream and write the output files so I can import them into my database.

I finally got through one of these 3mb files and tried the

System.out.println(line);

>    out.flush();

u can get rid of that to

and then the files do not get created... So somewhere in that part is where the files are being created.. what I think is happening when I change the code to this, instead of creating the files it displays the output on the screen..

J

0
 
LVL 92

Assisted Solution

by:objects
objects earned 1000 total points
ID: 16423574
nothing in that code creates a file, so what handles creating the file?
sorry for all the questions, just trying to work out whats doing what.
0
 
LVL 86

Accepted Solution

by:
CEHJ earned 1000 total points
ID: 16424291
If all that's happening is that you're being sent a file over TCP/IP that you want to save, simply copy the InputStream of the accepted Socket to a file
0
 
LVL 3

Author Comment

by:jsuttor
ID: 16504371
It seemed my problem was the reading of the stream... It was waiting for the stream to finish a file and then trying to read the entire stream to find the file name. Well if the stream was 2mb it was taking about 4 hours to read the stream when it only really needed to search the first 200 characters after the ?xml to find the file name. Once we switched the that a 2mb file took less than 30 seconds to write! Amazing! Thanks for you help both of you object and CEHJ.... you helped my friend narrow down the solution.
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16505100
:-)
0

Featured Post

Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Introduction This article is the second of three articles that explain why and how the Experts Exchange QA Team does test automation for our web site. This article covers the basic installation and configuration of the test automation tools used by…
In this post we will learn how to make Android Gesture Tutorial and give different functionality whenever a user Touch or Scroll android screen.
Viewers will learn about arithmetic and Boolean expressions in Java and the logical operators used to create Boolean expressions. We will cover the symbols used for arithmetic expressions and define each logical operator and how to use them in Boole…
This tutorial will introduce the viewer to VisualVM for the Java platform application. This video explains an example program and covers the Overview, Monitor, and Heap Dump tabs.
Suggested Courses
Course of the Month20 days, 21 hours left to enroll

810 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question