• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 202
  • Last Modified:

screen scraper

Does anybody have source code for a servlet screen scraper. the results needs to be stored in a database. E.g. get a particular book title from a web site.
0
Jaysri
Asked:
Jaysri
1 Solution
 
bloodredsunCommented:
Here's the code for a screen scraper, which results in a StringBuffer with the HTML in. You cn then parse out the code you want and then put it in the database using JDBC.

try {
                              //set up HttpURLConnection
                              URL u = new URL( urls[jj] );
                              HttpURLConnection huc = (HttpURLConnection) u.openConnection();
                              huc.setRequestMethod( "GET" );
                              huc.connect();
                              BufferedReader br = new BufferedReader(new InputStreamReader(huc.getInputStream()));

                              int b = 0;
                              while((b = br.read()) != -1) {
                                 sb.append((char)b);
                              }
                              br.close();
                              // disconnect HttpURLConnection
                              huc.disconnect() ;
                              System.out.println("done" + new Date());
                        }catch (IOException e){
                              System.out.println( "Unable to open connection: " + e.getMessage() );
                        }//end try/catch connection
0
 
JaysriAuthor Commented:
Thank you much..It worked!

Jay.
0

Featured Post

The new generation of project management tools

With monday.com’s project management tool, you can see what everyone on your team is working in a single glance. Its intuitive dashboards are customizable, so you can create systems that work for you.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now