[Webinar] Streamline your web hosting managementRegister Today

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 1137
  • Last Modified:

Problem trying to run wget from java program


Hello,

I try to run WGET to download XML file from one of our vendor. It's an HTTP site.

First, I try running it from command line and I get a lot of data (which is what I expected).
But when I try running it from the program, I got different data. Many times only the header of the XML. (Look at the content-length difference)

Can someone please advise what might be the cause of the problem? and how to solve this problem?
Thanks in advance for the help.


;--------------- RUNNING FROM COMMAND LINE ---------------;

wget -S -v -O test.xml 'http://......?paramA=1&paramB=2'

HTTP request sent, awaiting response...
  HTTP/1.1 200 OK
  Cache-Control: private, max-age=0
  Content-Type: text/xml; charset=utf-8
  Server: Microsoft-IIS/7.0
  X-AspNet-Version: 2.0.50727
  X-Powered-By: ASP.NET
  Date: Thu, 01 Apr 2010 16:31:47 GMT
  Connection: keep-alive
  Content-Length: 684207
Length: 684207 (668K) [text/xml]
Saving to: `test.xml'

100%[==============================================================================================>] 684,207      861K/s   in 0.8s

11:32:24 (861 KB/s) - `test.xml' saved [684207/684207]

;--------------- RUNNING FROM MY PROGRAM ---------------;

String myURL = "http://......?paramA=1&paramB=2"

String fileName = "test.xml";
String sh = "/bin/sh";
String cop = "-c";
String wget_command = "wget -S -v -O " + fileName + " '" + myURL + "'";
String[] exe = new String[] {sh, cop, wget_command};
Process process = Runtime.getRuntime().exec(exe);

HTTP request sent, awaiting response... 
  HTTP/1.1 200 OK
  Cache-Control: private, max-age=0
  Content-Type: text/xml; charset=utf-8
  Server: Microsoft-IIS/7.0
  X-AspNet-Version: 2.0.50727
  X-Powered-By: ASP.NET
  Date: Thu, 01 Apr 2010 16:30:20 GMT
  Connection: keep-alive
  Content-Length: 1425
Length: 1425 (1.4K) [text/xml]
Saving to: `test.xml'

     0K .                                                     100%  170M=0s

11:30:57 (170 MB/s) - `test.xml' saved [1425/1425]

Open in new window

0
rnicholus
Asked:
rnicholus
  • 4
  • 3
  • 3
  • +2
3 Solutions
 
cmalakarCommented:
How are you reading the stream, in java code ?
0
 
rnicholusAuthor Commented:
wget will download it to file.

and then i process the file.
0
 
cmalakarCommented:
What you do.. after the following line in java code ?

Process process = Runtime.getRuntime().exec(exe);

Probably completely code will help.
0
The new generation of project management tools

With monday.com’s project management tool, you can see what everyone on your team is working in a single glance. Its intuitive dashboards are customizable, so you can create systems that work for you.

 
rnicholusAuthor Commented:
Yes, it is. It is already shown in my original post.
0
 
cmalakarCommented:
Add the following lines.. after
Process process = Runtime.getRuntime().exec(exe);

int exitValue = process.waitFor();
System.out.println("Exit Value of WGET:" + exitValue);
0
 
CEHJCommented:
You'll need to do more than that probably, as you need to consume all streams. You might try cmalakar's suggestion with the following though
String wget_command = "wget -q -O " + fileName + " '" + myURL + "'";

Open in new window

0
 
CEHJCommented:
You can make your code stronger and more readable with
String[] wget_command = new String[] { "wget", "-q",  "-O", fileName, myURL };

Open in new window

0
 
JIEXACommented:
I'd first ensure that the URL in the script and the one in Java are absolutely equal. You know, special symbols, escape sequences, backslashes...
0
 
objectsCommented:
> You'll need to do more than that probably, as you need to consume all streams. You might try cmalakar's suggestion with the following though

sorry but how is that related to the problem?
0
 
JIEXACommented:
The difference in the "Content-Length: " tells that the content is different. Probably, you run the command from different directory/user, and cached cookies are different?

You can do "strace -f wget ..." and check which files it opens in both cases (command line and Java), and compare the results.
0
 
rnicholusAuthor Commented:
After further investigation,

Apparently there is something wrong that hinders data access using program.
It's the vendor's issue.

I will give points for the assistance.
Thank you guys!
0
 
rnicholusAuthor Commented:
Thank you for all the help!
0
 
CEHJCommented:
:-)
0

Featured Post

Learn to develop an Android App

Want to increase your earning potential in 2018? Pad your resume with app building experience. Learn how with this hands-on course.

  • 4
  • 3
  • 3
  • +2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now