• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 281
  • Last Modified:

URL HttpURLConnection return no response

The following Java code works okay as a standalone program but when I run it in a web application it has issues.

The issue is the web content returned doesn't have a http response.  It just returns binary content.  The http response is necessary because it gets forward to a browser using a proxy.  The browser can't interpret the content which is necessary for the application to work.

Any ideas what could be the problem?
package urlgetbytes;
 
import java.io.BufferedInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
 
/**
 *
 */
public class Main {
 
    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) throws MalformedURLException, IOException, InterruptedException {
        // TODO code application logic here
        URL url = new URL("http://localwebsite");
        HttpURLConnection uc = (HttpURLConnection) url.openConnection();
 
        uc.setRequestProperty("Host", "localwebsite");
        uc.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1");
        uc.setRequestProperty("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
        uc.setRequestProperty("Accept-Language", "en-us,en;q=0.5");
        uc.setRequestProperty("Accept-Encoding", "gzip,deflate");
        uc.setRequestProperty("Accept-Charset", "ISO-8859-1,utf-8;q=0.7,*;q=0.7");
        uc.setRequestProperty("Keep-Alive", "300");
        uc.setRequestProperty("Connection", "keep-alive");
 
        uc.connect();
        InputStream in = uc.getInputStream();
 
        ByteArrayOutputStream out = new ByteArrayOutputStream();
 
 
        byte[] b = new byte[1024];
        int i = in.read(b);
        while (i != -1) {
            out.write(b, 0, i);
            i = in.read(b);
            System.out.println("i: " + i);
        }
        out.close();
        in.close();
 
        byte[] response = out.toByteArray();
    }
}

Open in new window

0
lcor
Asked:
lcor
  • 7
  • 6
1 Solution
 
objectsCommented:
how are you running it in your webapp?

why aren't you using the url class?

http://helpdesk.objects.com.au/java/how-to-write-get-response-to-byte-array
0
 
lcorAuthor Commented:
I am using the URL class.

See first line:
URL url = new URL("http://localwebsite");
0
 
lcorAuthor Commented:
I did a test.  I created a standalone web application and placed this code in a servlet.  I'm finding I get no http response.

I created a test java application (not web) and placed the same exact code there.  I get an http response.

Why does the servlet affect this code?
0
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

 
objectsCommented:
what exactly do you mean by no response? have you checked the respomnse code?

are any proxy servers involved?

0
 
lcorAuthor Commented:
yes, there's a proxy server so that's why I have to get the content.   Here's the flow.  The browser is configured to use a proxy

above code -> java client -> proxy -> browser

I got the response using get Headers and placed the HTTP OK header in the byte array.  The result is the  content that finally gets to the browser is incorrect.  The browser then reports an compression error.

There must be something wrong with the above code.

0
 
objectsCommented:
whats the browser got to do with it?

0
 
lcorAuthor Commented:
Firefox
0
 
lcorAuthor Commented:
the client sends the content gotten from the above code to the browser.  

If you were run the above code, examine the content with wireshark, with actual content gotten by the browser alone, you will see they are not same.
0
 
objectsCommented:
how is it different?

0
 
lcorAuthor Commented:
The content returned by url class is much larger than the real line signature.
0
 
objectsCommented:
i need more details from you to help, did you try the code I posted in the link above?

0
 
lcorAuthor Commented:
Yes, I tried that..the problem for my case is that I need the exact "type" of content returned that the browser would get.  I can't just use the code you posted alone.  I have to set request properties.  I took the exact GET properties for a real website and set request properties.  I then added them to your code.  The results do not return the same content a browser returns.  In fact, the content was somewhat strange.  The url class tended to get more bytes than needed. I tried the url class and setting request properties with multiple websites.  I had to revert to the socket class in order to get the same type behavior a browser gets.
0
 
objectsCommented:
That won't work as the behaviour is typically dynamic
try HttpClient, it gives you greater control over what is sent.

0

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

  • 7
  • 6
Tackle projects and never again get stuck behind a technical roadblock.
Join Now