Solved

How can I configure a java webcrawler to pull RSS data into a database?

Posted on 2007-11-30
6
192 Views
Last Modified: 2008-02-01
I would like to configure a java webcrawler to read news all day from specific news sites and put the news titles and body into a database for analysis by another application.  What would be the easiest way to work this out?

Ideally, I would create a webcrawler object with a constructor that takes an address of a RSS feed, and then call methods on the object that return the contents of the feed as strings or streams or anything that will allow me to parse and manipulate the data into a database or for some other reason.  I have done the basic searches on the topic and most of the information I found about it was for JSP and contained code to reproduce html pages.  I dont really need this, I just need the information from news sites in a java environment where I can run java code on it.

thanks!
-md
0
Comment
Question by:meuedyn
  • 3
  • 2
6 Comments
 
LVL 86

Accepted Solution

by:
CEHJ earned 500 total points
ID: 20386349
One or more of these should be fine:

http://java-source.net/open-source/rss-rdf-tools
0
 
LVL 92

Expert Comment

by:objects
ID: 20386644
0
 
LVL 9

Expert Comment

by:ysnky
ID: 20387550
0
Optimizing Cloud Backup for Low Bandwidth

With cloud storage prices going down a growing number of SMBs start to use it for backup storage. Unfortunately, business data volume rarely fits the average Internet speed. This article provides an overview of main Internet speed challenges and reveals backup best practices.

 
LVL 86

Expert Comment

by:CEHJ
ID: 20387665
>>
try informa
...
or feedparser
>>

(both already mentioned)
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 20397586
:-)
0
 
LVL 92

Expert Comment

by:objects
ID: 20398774
thought you would have liked a little more insight than someone googling just an answer for you. Next time just type "java rss api" in google and save yourself some time :)
0

Featured Post

Are your AD admin tools letting you down?

Managing Active Directory can get complicated.  Often, the native tools for managing AD are just not up to the task.  The largest Active Directory installations in the world have relied on one tool to manage their day-to-day administration tasks: Hyena. Start your trial today.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Java Server Faces parameter pass? 6 50
Java DateChooser? 3 36
stackato and cloud 4 85
Java string replace 11 45
INTRODUCTION Working with files is a moderately common task in Java.  For most projects hard coding the file names, using parameters in configuration files, or using command-line arguments is sufficient.   However, when your application has vi…
Java had always been an easily readable and understandable language.  Some relatively recent changes in the language seem to be changing this pretty fast, and anyone that had not seen any Java code for the last 5 years will possibly have issues unde…
Viewers will learn about basic arrays, how to declare them, and how to use them. Introduction and definition: Declare an array and cover the syntax of declaring them: Initialize every index in the created array: Example/Features of a basic arr…
This tutorial covers a practical example of lazy loading technique and early loading technique in a Singleton Design Pattern.

770 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question