?
Solved

Wget and useragent options in java

Posted on 2011-09-08
4
Medium Priority
?
383 Views
Last Modified: 2013-11-23
I want convert the below linux command into java a java program. How can i set useragent option in java??

>>>> wget www.somesite.com -U "option1 option2"

Note : here -U is useragent options



0
Comment
Question by:manugeek
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
4 Comments
 
LVL 86

Assisted Solution

by:CEHJ
CEHJ earned 200 total points
ID: 36501539
Set http.agent System property. You can also set it as a header in URLConnection, but only the former is necessary
0
 
LVL 47

Accepted Solution

by:
for_yan earned 800 total points
ID: 36503892


This gives you some variants of code for wget-like Java programs:
http://stackoverflow.com/questions/238547/how-do-you-programmatically-download-a-webpage-in-java
and this is the recommendation for user agents from the last answer:

URL url = new URL(urlStr);
HttpURLConnection conn = (HttpURLConnection) url.openConnection(); // Cast shouldn't fail
HttpURLConnection.setFollowRedirects(true);
// allow both GZip and Deflate (ZLib) encodings
conn.setRequestProperty("Accept-Encoding", "gzip, deflate");
String encoding = conn.getContentEncoding();
InputStream inStr = null;

// create the appropriate stream wrapper based on
// the encoding type
if (encoding != null && encoding.equalsIgnoreCase("gzip")) {
    inStr = new GZIPInputStream(conn.getInputStream());
} else if (encoding != null && encoding.equalsIgnoreCase("deflate")) {
    inStr = new InflaterInputStream(conn.getInputStream(),
      new Inflater(true));
} else {
    inStr = conn.getInputStream();
}

Open in new window


User- agent:

conn.setRequestProperty ( "User-agent", "my agent name");

0
 
LVL 86

Expert Comment

by:CEHJ
ID: 36509531
Better to use the first solution i mentioned: you can pass it to the runtime with the ability to change it at will without recompilation
0
 

Author Comment

by:manugeek
ID: 36509728
Thanks CEHJ, it was helpful. I am rookie to java but will appreciate any working example or a link. Sorry i will not be able to give any more points has this question is marked closed.
0

Featured Post

Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Since upgrading to Office 2013 or higher installing the Smart Indenter addin will fail. This article will explain how to install it so it will work regardless of the Office version installed.
This article will show how Aten was able to supply easy management and control for Artear's video walls and wide range display configurations of their newsroom.
Viewers will learn about the regular for loop in Java and how to use it. Definition: Break the for loop down into 3 parts: Syntax when using for loops: Example using a for loop:
Starting up a Project
Suggested Courses
Course of the Month7 days, 23 hours left to enroll

765 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question