Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17


How can I configure a java webcrawler to pull RSS data into a database?

Posted on 2007-11-30
Medium Priority
Last Modified: 2008-02-01
I would like to configure a java webcrawler to read news all day from specific news sites and put the news titles and body into a database for analysis by another application.  What would be the easiest way to work this out?

Ideally, I would create a webcrawler object with a constructor that takes an address of a RSS feed, and then call methods on the object that return the contents of the feed as strings or streams or anything that will allow me to parse and manipulate the data into a database or for some other reason.  I have done the basic searches on the topic and most of the information I found about it was for JSP and contained code to reproduce html pages.  I dont really need this, I just need the information from news sites in a java environment where I can run java code on it.

Question by:meuedyn
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
LVL 86

Accepted Solution

CEHJ earned 2000 total points
ID: 20386349
One or more of these should be fine:
LVL 92

Expert Comment

ID: 20386644
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

LVL 86

Expert Comment

ID: 20387665
try informa
or feedparser

(both already mentioned)
LVL 86

Expert Comment

ID: 20397586
LVL 92

Expert Comment

ID: 20398774
thought you would have liked a little more insight than someone googling just an answer for you. Next time just type "java rss api" in google and save yourself some time :)

Featured Post

Build and deliver software with DevOps

A digital transformation requires faster time to market, shorter software development lifecycles, and the ability to adapt rapidly to changing customer demands. DevOps provides the solution.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

For beginner Java programmers or at least those new to the Eclipse IDE, the following tutorial will show some (four) ways in which you can import your Java projects to your Eclipse workbench. Introduction While learning Java can be done with…
Java contains several comparison operators (e.g., <, <=, >, >=, ==, !=) that allow you to compare primitive values. However, these operators cannot be used to compare the contents of objects. Interface Comparable is used to allow objects of a cl…
The viewer will learn how to implement Singleton Design Pattern in Java.
This tutorial will introduce the viewer to VisualVM for the Java platform application. This video explains an example program and covers the Overview, Monitor, and Heap Dump tabs.
Suggested Courses

661 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question