Solved

How can I configure a java webcrawler to pull RSS data into a database?

Posted on 2007-11-30
6
190 Views
Last Modified: 2008-02-01
I would like to configure a java webcrawler to read news all day from specific news sites and put the news titles and body into a database for analysis by another application.  What would be the easiest way to work this out?

Ideally, I would create a webcrawler object with a constructor that takes an address of a RSS feed, and then call methods on the object that return the contents of the feed as strings or streams or anything that will allow me to parse and manipulate the data into a database or for some other reason.  I have done the basic searches on the topic and most of the information I found about it was for JSP and contained code to reproduce html pages.  I dont really need this, I just need the information from news sites in a java environment where I can run java code on it.

thanks!
-md
0
Comment
Question by:meuedyn
  • 3
  • 2
6 Comments
 
LVL 86

Accepted Solution

by:
CEHJ earned 500 total points
Comment Utility
One or more of these should be fine:

http://java-source.net/open-source/rss-rdf-tools
0
 
LVL 92

Expert Comment

by:objects
Comment Utility
0
 
LVL 9

Expert Comment

by:ysnky
Comment Utility
0
How to improve team productivity

Quip adds documents, spreadsheets, and tasklists to your Slack experience
- Elevate ideas to Quip docs
- Share Quip docs in Slack
- Get notified of changes to your docs
- Available on iOS/Android/Desktop/Web
- Online/Offline

 
LVL 86

Expert Comment

by:CEHJ
Comment Utility
>>
try informa
...
or feedparser
>>

(both already mentioned)
0
 
LVL 86

Expert Comment

by:CEHJ
Comment Utility
:-)
0
 
LVL 92

Expert Comment

by:objects
Comment Utility
thought you would have liked a little more insight than someone googling just an answer for you. Next time just type "java rss api" in google and save yourself some time :)
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

Suggested Solutions

Title # Comments Views Activity
Issues installing SSL certificate into Apache Tomcat 3 70
countAbc challenge 9 49
wordcount challenge 11 74
Problem to start Neon 20 49
For beginner Java programmers or at least those new to the Eclipse IDE, the following tutorial will show some (four) ways in which you can import your Java projects to your Eclipse workbench. Introduction While learning Java can be done with…
Java Flight Recorder and Java Mission Control together create a complete tool chain to continuously collect low level and detailed runtime information enabling after-the-fact incident analysis. Java Flight Recorder is a profiling and event collectio…
Viewers learn about the third conditional statement “else if” and use it in an example program. Then additional information about conditional statements is provided, covering the topic thoroughly. Viewers learn about the third conditional statement …
This theoretical tutorial explains exceptions, reasons for exceptions, different categories of exception and exception hierarchy.

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now