screen scraper

Posted on 2005-03-08
Medium Priority
Last Modified: 2006-11-17
Does anybody have source code for a servlet screen scraper. the results needs to be stored in a database. E.g. get a particular book title from a web site.
Question by:Jaysri
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
LVL 29

Accepted Solution

bloodredsun earned 2000 total points
ID: 13494283
Here's the code for a screen scraper, which results in a StringBuffer with the HTML in. You cn then parse out the code you want and then put it in the database using JDBC.

try {
                              //set up HttpURLConnection
                              URL u = new URL( urls[jj] );
                              HttpURLConnection huc = (HttpURLConnection) u.openConnection();
                              huc.setRequestMethod( "GET" );
                              BufferedReader br = new BufferedReader(new InputStreamReader(huc.getInputStream()));

                              int b = 0;
                              while((b = br.read()) != -1) {
                              // disconnect HttpURLConnection
                              huc.disconnect() ;
                              System.out.println("done" + new Date());
                        }catch (IOException e){
                              System.out.println( "Unable to open connection: " + e.getMessage() );
                        }//end try/catch connection

Author Comment

ID: 13502387
Thank you much..It worked!


Featured Post

On Demand Webinar: Networking for the Cloud Era

Did you know SD-WANs can improve network connectivity? Check out this webinar to learn how an SD-WAN simplified, one-click tool can help you migrate and manage data in the cloud.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

After being asked a question last year, I went into one of my moods where I did some research and code just for the fun and learning of it all.  Subsequently, from this journey, I put together this article on "Range Searching Using Visual Basic.NET …
In this post we will learn how to make Android Gesture Tutorial and give different functionality whenever a user Touch or Scroll android screen.
Viewers will learn about basic arrays, how to declare them, and how to use them. Introduction and definition: Declare an array and cover the syntax of declaring them: Initialize every index in the created array: Example/Features of a basic arr…
This tutorial covers a practical example of lazy loading technique and early loading technique in a Singleton Design Pattern.
Suggested Courses
Course of the Month11 days, 10 hours left to enroll

752 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question