Tolgar
asked on
How to re-write this code with httpclient and/or httpURLconnection instead of httpunit?
Hi,
I have the following code which uses com.meterware.httpunit package.
How can I rewrite this code using httpclient and/or httpURLconnection instead of httpunit package?
Note: This code parses 2nd table in the web page and generates something like:
text1 --> texta
text2 --> textb
text3 --> textc
The source of this page looks like this:
I have the following code which uses com.meterware.httpunit package.
How can I rewrite this code using httpclient and/or httpURLconnection instead of httpunit package?
Note: This code parses 2nd table in the web page and generates something like:
text1 --> texta
text2 --> textb
text3 --> textc
The source of this page looks like this:
* <tr>
* <td><a href="somepath/check.html#check_del_dir">del_dir</a>
* <td><a href="report.check.log.html#del_dir"><font color=green>pass</font></a>
* <tr>
* <td><a href="somepath/check.html#check_type">type</a>
* <td><a href="report.check.log.html#type"><font color=red>fail</font></a>
try {
HttpUnitOptions.setScriptingEnabled(false);
WebConversation wc = new WebConversation();
// Check if the URL is valid
if (checkReportPath.startsWith("http://") && (checkReportPath.indexOf("/check/report.html") > -1)){
WebResponse wr = wc.getResponse(checkReportPath);
WebTable table = wr.getTables()[2];
java.lang.String[][] cells = table.asText();
for(java.lang.String[] row : cells) {
boolean column = true;
for(java.lang.String cell : row) {
if (column){
checkActivity.append(cell).append(" --> ");
column = false;
}
else {
checkActivity.append(cell);
}
}
checkActivity.append("\n");
}
}
}
I guess, your main point is how to parse the tables.
And HttpUnit does provide some conveniences there, that is true,
but with HttpUnit I recently experienced OutOfMemory
errors after some time of operation,
and I had to switch to standard Java classes.
There is a number of HTML parsers around, perhaps they can help to manipulate
the output and get table data; there was recently a question with a list
of such parsers. Let me try to find it.
And HttpUnit does provide some conveniences there, that is true,
but with HttpUnit I recently experienced OutOfMemory
errors after some time of operation,
and I had to switch to standard Java classes.
There is a number of HTML parsers around, perhaps they can help to manipulate
the output and get table data; there was recently a question with a list
of such parsers. Let me try to find it.
ASKER
Sorry, I couldn't get why you used yahoo.openConnection in this example.
Thanks,
Thanks,
ASKER
Yes, I need to parse the second table. If you can find a better parser it would be great.
Thanks,
Thanks,
I'm sorry:
yahoo.openConnection()
should of course be changed to
my_source.openConnection()
in this context (I changed in one place but noty in another)
I'm still looking for this question where there were links to many parsers, I should find it
yahoo.openConnection()
should of course be changed to
my_source.openConnection()
in this context (I changed in one place but noty in another)
I'm still looking for this question where there were links to many parsers, I should find it
For some reason cannot find that question (I never can find anything using EE search - don't know why),
but these are some links with discussions of different parsers:
http://stackoverflow.com/questions/3152138/what-are-the-pros-and-cons-of-the-leading-java-html-parsers
http://www.benmccann.com/dev-blog/java-html-parsing-library-comparison/
Why do you want to go away from HttpUnit - are you experiencing something similar
which I had with OutOfMemory issue?
ASKER
No. It is just a design decision. The reason is it may overload the code with some unnecessary stuff. It looks like it was written for unit testing.
OK, then you probably don't want to use any parser, and want to somehow
parse it yourself.
I actually thought that parsing piece of HTttpUnit was fine, and I wouyld have been happy with it, but I had
this issue with OutOfMemory
Why do you want to avoid HttpUnit?
ASKER
It was just a design decision.
I thought it may be so heavy for just parsing a source code/
Thanks,
I thought it may be so heavy for just parsing a source code/
Thanks,
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
This is how it was (I now commented it out):
Open in new window
This is how it is now without HttpUnit:
Open in new window