Solved

How can I configure a java webcrawler to pull RSS data into a database?

Posted on 2007-11-30
6
193 Views
Last Modified: 2008-02-01
I would like to configure a java webcrawler to read news all day from specific news sites and put the news titles and body into a database for analysis by another application.  What would be the easiest way to work this out?

Ideally, I would create a webcrawler object with a constructor that takes an address of a RSS feed, and then call methods on the object that return the contents of the feed as strings or streams or anything that will allow me to parse and manipulate the data into a database or for some other reason.  I have done the basic searches on the topic and most of the information I found about it was for JSP and contained code to reproduce html pages.  I dont really need this, I just need the information from news sites in a java environment where I can run java code on it.

thanks!
-md
0
Comment
Question by:meuedyn
  • 3
  • 2
6 Comments
 
LVL 86

Accepted Solution

by:
CEHJ earned 500 total points
ID: 20386349
One or more of these should be fine:

http://java-source.net/open-source/rss-rdf-tools
0
 
LVL 92

Expert Comment

by:objects
ID: 20386644
0
 
LVL 9

Expert Comment

by:ysnky
ID: 20387550
0
Free Tool: Postgres Monitoring System

A PHP and Perl based system to collect and display usage statistics from PostgreSQL databases.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

 
LVL 86

Expert Comment

by:CEHJ
ID: 20387665
>>
try informa
...
or feedparser
>>

(both already mentioned)
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 20397586
:-)
0
 
LVL 92

Expert Comment

by:objects
ID: 20398774
thought you would have liked a little more insight than someone googling just an answer for you. Next time just type "java rss api" in google and save yourself some time :)
0

Featured Post

Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
ejb wildfly example 2 29
Adding multiple JVM environments to RedHat 6 7 49
Tagging and Merging on Branch 1 30
Running JavaFX on the Raspberry Pi 27 48
Java Flight Recorder and Java Mission Control together create a complete tool chain to continuously collect low level and detailed runtime information enabling after-the-fact incident analysis. Java Flight Recorder is a profiling and event collectio…
Introduction This article is the first of three articles that explain why and how the Experts Exchange QA Team does test automation for our web site. This article explains our test automation goals. Then rationale is given for the tools we use to a…
This theoretical tutorial explains exceptions, reasons for exceptions, different categories of exception and exception hierarchy.
This tutorial will introduce the viewer to VisualVM for the Java platform application. This video explains an example program and covers the Overview, Monitor, and Heap Dump tabs.

821 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question