Pull Data from Websites

Hi All,

I am trying to pull feeds from websites, like for new  homes from both websites, twitter and social media.  Does anyone know an easy way to pull this type of data and check for updates every hour or so?  Also, would like to import into a database.....

Thanks in advance

Who is Participating?

[Webinar] Streamline your web hosting managementRegister Today

Scott Fell, EE MVEConnect With a Mentor DeveloperCommented:
>i just want to download some data from public sites....for new  homes.....

That type of data is going to be tightly controlled. You are going to need a 3rd party service like http://www.idxbroker.com/

Creating a twitter feed should not be too hard, but you will need to define what you want  https://twitter.com/search?q=%23newhomes&src=typd and if that is the type of data you are after.
Scott Fell, EE MVEConnect With a Mentor DeveloperCommented:
Yes, check their api documentation and terms of service first.

One api could be http://www.zillow.com/howto/api/PropertyDetailsAPIOverview.htm. For any api you use, be sure to note the TOS http://www.zillow.com/howto/api/APITerms.htm it will explain the usage data limits.  

If you are talking about your website crawling another website and extract data, that is called screen scraping and for the most part is not legal as it probably violates copy rights.  For any homes type of site that is generated by a MLS (Multiple Listing Service) from a RealtorĀ®, that data is protected.

If you can point to a specific site you want to use and the data is open source for your use, we can help give you guidance but using an api from something like zillow is going to be the best way to go.

There is a way to use RealtorĀ® data through services like http://ww.idxbroker.com
Praveen DMConnect With a Mentor Infra Team LeadCommented:
I have come across an old process , XML feed pull and populating them in DB .I don't remember the complete process though.

But the XML feed update is the free and most used across sites.
[Webinar] Improve your customer journey

A positive customer journey is important in attracting and retaining business. To improve this experience, you can use Google Maps APIs to increase checkout conversions, boost user engagement, and optimize order fulfillment. Learn how in this webinar presented by Dito.

Scott Fell, EE MVEConnect With a Mentor DeveloperCommented:
servoadmin, your answers are about creating a feed.  That is not what Jack_son_ is asking about.
Jack_son_Author Commented:
i just want to download some data from public sites and allow it to update as needed; is there a tool that can accomplish this and put it in a database?
All Courses

From novice to tech pro — start learning today.