Forum Post Grabber/organizer

Hello experts,

I am looking for a good solution (software) that will help me archive questions and replies from certain forums (or RSS feed) and put them into my database structure for offline viewing and search.

It must be easy with RSS but I am not sure if the forums are using a global format which will allow a software to grab information in them.

Please give me recommendations if such software exists?

Thanks

[No RSS readers please, I used a few of them but they are not what I am looking for]
bozerAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Jason C. LevineNo oneCommented:
Hi bozer,

There are a wide variety of spidering programs out there if you Google for them...
0
bozerAuthor Commented:
Hi Jason,

I did several times but maybe I am not using the correct words. I always get lots of RSS Readers but nothing about forum entries reader or exporter into DB

Thanks
0
Jason C. LevineNo oneCommented:
Try searching for "offline website downloader" instead.   That should produce a ton of usable results.

Just make sure you run this on sites that permit it.   Sometimes copying the contents offline is prohibited by the Terms of Service.
0
Cloud Class® Course: Certified Penetration Testing

This CPTE Certified Penetration Testing Engineer course covers everything you need to know about becoming a Certified Penetration Testing Engineer. Career Path: Professional roles include Ethical Hackers, Security Consultants, System Administrators, and Chief Security Officers.

bozerAuthor Commented:
But it's not what I want,

I need a software that will parse the questions and replies and put them into a database.

An offline website download will not do that.
0
Jason C. LevineNo oneCommented:
I see.

I'm not sure what you are asking for exists as commercial software.  There are too many variables from site to site to have one product fit all.

In this situation, you are looking at writing a scraper script that is aimed at a specific site.  Once you limit yourself to a single site, you can analyze the page structure and determine the HTML containers.  From there, a simple regular expression extracts what you want.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
bozerAuthor Commented:
Hello,

I was hoping there are already prepared applications for popular Forum platforms like Invision, etc.

I guess there is not any?
0
Jason C. LevineNo oneCommented:
Not really, since what you are asking for goes against the reason for people to put up a board in the first place, namely to get folks on the site.
0
Jason C. LevineNo oneCommented:
My last two posts provide a long-winded version of "what you want doesn't exist" and should be saved as a solution for future readers.  I'm willing to not accept any points for the solution since I do agree that it isn't very helpful to the original asker.

0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Software

From novice to tech pro — start learning today.