How can I configure a java webcrawler to pull RSS data into a database?

Posted on 2007-11-30
Last Modified: 2008-02-01
I would like to configure a java webcrawler to read news all day from specific news sites and put the news titles and body into a database for analysis by another application.  What would be the easiest way to work this out?

Ideally, I would create a webcrawler object with a constructor that takes an address of a RSS feed, and then call methods on the object that return the contents of the feed as strings or streams or anything that will allow me to parse and manipulate the data into a database or for some other reason.  I have done the basic searches on the topic and most of the information I found about it was for JSP and contained code to reproduce html pages.  I dont really need this, I just need the information from news sites in a java environment where I can run java code on it.

Question by:meuedyn
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
LVL 86

Accepted Solution

CEHJ earned 500 total points
ID: 20386349
One or more of these should be fine:
LVL 92

Expert Comment

ID: 20386644

Expert Comment

ID: 20387550
Instantly Create Instructional Tutorials

Contextual Guidance at the moment of need helps your employees adopt to new software or processes instantly. Boost knowledge retention and employee engagement step-by-step with one easy solution.

LVL 86

Expert Comment

ID: 20387665
try informa
or feedparser

(both already mentioned)
LVL 86

Expert Comment

ID: 20397586
LVL 92

Expert Comment

ID: 20398774
thought you would have liked a little more insight than someone googling just an answer for you. Next time just type "java rss api" in google and save yourself some time :)

Featured Post

On Demand Webinar - Networking for the Cloud Era

This webinar discusses:
-Common barriers companies experience when moving to the cloud
-How SD-WAN changes the way we look at networks
-Best practices customers should employ moving forward with cloud migration
-What happens behind the scenes of SteelConnect’s one-click button

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Introduction This article is the last of three articles that explain why and how the Experts Exchange QA Team does test automation for our web site. This article covers our test design approach and then goes through a simple test case example, how …
Java functions are among the best things for programmers to work with as Java sites can be very easy to read and prepare. Java especially simplifies many processes in the coding industry as it helps integrate many forms of technology and different d…
Viewers learn about the “while” loop and how to utilize it correctly in Java. Additionally, viewers begin exploring how to include conditional statements within a while loop and avoid an endless loop. Define While Loop: Basic Example: Explanatio…
This tutorial covers a step-by-step guide to install VisualVM launcher in eclipse.
Suggested Courses

752 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question