Solved

How to get more RSS feed items from reuters

Posted on 2009-05-12
4
545 Views
Last Modified: 2013-11-18
We are trying to display relevant news items on our ASP.NET site. We download these feed items from reuters. for example, we use the link http://feeds.reuters.com/reuters/topNews?format=xml

The problem is, the reuters feed is giving only the latest 10 items and the older ones are quickly expiring. We would like to get all the news items of the last month or last 7 days depending on the preferences. Is there someway to get this done?
0
Comment
Question by:shore-support
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
4 Comments
 

Expert Comment

by:jones1618
ID: 24375498
Two answers spring to mind:

1. Locally cache Reuters news items yourself. If yours is a high traffic site, that's just the courteous thing to do.

Pseudo-code: Check cache age. If it is older than X hours, fetch latest RSS from Reuters. Add new items (if any) to cache. Expire any items older than X days/hours.

2. Reuters Labs provides a richer set of feed options including a "count" parameter (which doesn't work in the feedburner feed, unfortunately.) Note: You have to register and declare your intended use of the data.

Reuters Labs - Spotlight
http://spotlight.reuters.com/page/2007/07/10/feeds
0
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 24408201
Here is how I might approach it...

Use a cron job to read the Reuters feed.  You can tinker with the interval - perhaps run this every few minutes, depending on the rate of change in the Reuters feed.

Pull the items out of the feed.  Make a hash code from the entire text of the item from the open <item> to the close </item>  Insert a timestamp, the hash code and the item text into your data base.  I would mark the hash code UNIQUE to force an error when you try to insert the same item twice.

If you're getting a lot of duplicate hash code errors you might lengthen the time between runs of the cron job.

Then use your data base to serve the items in your RSS feed.

HTH, ~Ray
0
 

Author Comment

by:shore-support
ID: 24421164
We want to use the feed items for commercial purpose. But the "Reuters Labs - Spotlight" prohibits use of feeds for commercial purposes.

I think the best way is to keep running our scheduled job at some interval and catch the feed items.

Google reader keeps all the items in cache. But to access those items, one needs to authenticate into the google services using google API. I think this API requires entering of captcha code by the user. If we can authenticate some how into GOOGLE Reader, then we can read the feed items. This way we do not need to cache the items.

Please let me know your comments?
0
 
LVL 110

Accepted Solution

by:
Ray Paseur earned 500 total points
ID: 24422899
I've used other Google services and have never needed a CAPTCHA code to use their API.  You access these services via a RESTful interface, passing the API key in the GET string of the URL.

I am not familiar with using Google Reader to access Reuters RSS feeds, but I recall another question in the RSS Zone from someone who found a similar situation (only a few recent feed items) in Google.

RSS feeds cost money to build and serve, and there is little added value in keeping lots of them around, especially when they are not making a company any money.  So for that reason I would think that the trend would be toward shorter feeds, and I would think that if you need more life of the items in the feed, you would be in the best position if you relied on your own ability to store and server the feeds.

My $0.02.  Hope that helps, ~Ray
0

Featured Post

Free Tool: Path Explorer

An intuitive utility to help find the CSS path to UI elements on a webpage. These paths are used frequently in a variety of front-end development and QA automation tasks.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
CSS Question.. 3 97
Where do I learn ES6? 3 81
JavaScript and Regular Expressions - How to implement date validation? 7 105
Gmail hacked or spoofed? 10 41
Introduction Since I wrote the original article about Handling Date and Time in PHP and MySQL (http://www.experts-exchange.com/articles/201/Handling-Date-and-Time-in-PHP-and-MySQL.html) several years ago, it seemed like now was a good time to updat…
SASS allows you to treat your CSS code in a more OOP way. Let's have a look on how you can structure your code in order for it to be easily maintained and reused.
Video by: Mark
This lesson goes over how to construct ordered and unordered lists and how to create hyperlinks.
Learn how to create flexible layouts using relative units in CSS.  New relative units added in CSS3 include vw(viewports width), vh(viewports height), vmin(minimum of viewports height and width), and vmax (maximum of viewports height and width).

763 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question