Java Socket Program Hangs

This Java program acquires web content from a website.  I'm having issues with msnbc.com.  After the 24th loop of acquiring content the progam hangs.

So after about the 24th i:, the program stops and then, eventually the socket resets.

Any ideas what could be the problem?  Any suggestions to fix?
OutputStream os = null;
        ByteArrayOutputStream byteContainer = new ByteArrayOutputStream();
        try {
            Socket socket = new Socket("www.msnbc.msn.com", 80);
 
            String r = new String("GET / HTTP/1.1\n" +
                    "Host: www.msnbc.msn.com\n" +
                    "User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12\n" +
                    "Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5\n" +
                    "Accept-Language: en-us,en;q=0.5\n" +
                    "Accept-Encoding: gzip,deflate\n" +
                    "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\n" +
                    "Keep-Alive: 300\n" +
                    "Connection: keep-alive\n" +
                    "Cookie: MC1=GUID=e72def902864496fa1074688e1697ab7; TZM=-240; __g_c=w%3A0; PageNo=1; s_cc=true; s_sq=%5B%5BB%5D%5D; MUID=F496E73B351C49BF8120BEECED8B6A98\n");
 
            System.out.println("Connecting using the following get request: " + r);
 
            os = socket.getOutputStream();
            boolean autoflush = true;
            PrintWriter out = new PrintWriter(socket.getOutputStream(), autoflush);
            out.println(new String(r));
            out.println('\r');
            out.println('\n');
            out.flush();
            socket.shutdownOutput();
 
            // Read the response
            byte[] b = new byte[1024];
            InputStream in = socket.getInputStream();
            int i = in.read(b);
            while (i != -1) {
                byteContainer.write(b, 0, i);
                i = in.read(b);
                System.out.println("i: " + i);
            }
 
            socket.close();
            in.close();
            out.close();
            os.close();
 
            byte[] content = byteContainer.toByteArray();
            byteContainer.close();
 
        } catch (IOException ex) {
            ex.printStackTrace();
        }

Open in new window

lcorAsked:
Who is Participating?
 
objectsConnect With a Mentor Commented:
0
 
Thomas4019Connect With a Mentor Commented:
Maybe their website is categorizing you calls as excessive. Maybe try waiting for a few hundred milliseconds between each call.
0
 
lcorAuthor Commented:
Thomas4019,
I added a Thread.sleep(300) in the while loop.

Each i comes back with a value of 1024 but it eventually ends up hanging.
0
 
n_sachin1Connect With a Mentor Commented:
IMHO, the read method of an inputstream is supposed to block till the end of stream is reached or there was an IO error/exception, if there is no more data available to be read. Thats what is happening in your program.

With HTTP1.1 calls, that have persistent connections, the server does not drop the connection after writing the data, but instead, sends information in the packets that the current response is complete(via content-length, etc.). You should look towards using the HTTP classes instead of plain Sockets, as they are higher level abstractions and understand the HTTP packets better.

Use java.net.HttpURLConnection as the entry point, or use the Apache Http Client library which is simpler to use.



0
 
lcorAuthor Commented:
n_sachin1 and objects are the solution but Thomas4019 was helpful with debug
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.