Solved

Pull Data from Websites

Posted on 2014-02-09
6
224 Views
Last Modified: 2014-03-06
Hi All,

I am trying to pull feeds from websites, like for new  homes from both websites, twitter and social media.  Does anyone know an easy way to pull this type of data and check for updates every hour or so?  Also, would like to import into a database.....

Thanks in advance

Jackson
0
Comment
Question by:Jack_son_
  • 3
  • 2
6 Comments
 
LVL 52

Assisted Solution

by:Scott Fell, EE MVE
Scott Fell,  EE MVE earned 300 total points
ID: 39846398
Yes, check their api documentation and terms of service first.

One api could be http://www.zillow.com/howto/api/PropertyDetailsAPIOverview.htm. For any api you use, be sure to note the TOS http://www.zillow.com/howto/api/APITerms.htm it will explain the usage data limits.  

If you are talking about your website crawling another website and extract data, that is called screen scraping and for the most part is not legal as it probably violates copy rights.  For any homes type of site that is generated by a MLS (Multiple Listing Service) from a Realtor®, that data is protected.

If you can point to a specific site you want to use and the data is open source for your use, we can help give you guidance but using an api from something like zillow is going to be the best way to go.

There is a way to use Realtor® data through services like http://ww.idxbroker.com
0
 
LVL 13

Assisted Solution

by:Praveen DM
Praveen DM earned 200 total points
ID: 39847431
I have come across an old process , XML feed pull and populating them in DB .I don't remember the complete process though.

But the XML feed update is the free and most used across sites.
0
 
LVL 13

Assisted Solution

by:Praveen DM
Praveen DM earned 200 total points
ID: 39847439
0
Use Case: Protecting a Hybrid Cloud Infrastructure

Microsoft Azure is rapidly becoming the norm in dynamic IT environments. This document describes the challenges that organizations face when protecting data in a hybrid cloud IT environment and presents a use case to demonstrate how Acronis Backup protects all data.

 
LVL 52

Assisted Solution

by:Scott Fell, EE MVE
Scott Fell,  EE MVE earned 300 total points
ID: 39847500
servoadmin, your answers are about creating a feed.  That is not what Jack_son_ is asking about.
0
 

Author Comment

by:Jack_son_
ID: 39863606
i just want to download some data from public sites and allow it to update as needed; is there a tool that can accomplish this and put it in a database?
0
 
LVL 52

Accepted Solution

by:
Scott Fell,  EE MVE earned 300 total points
ID: 39863747
>i just want to download some data from public sites....for new  homes.....

That type of data is going to be tightly controlled. You are going to need a 3rd party service like http://www.idxbroker.com/

Creating a twitter feed should not be too hard, but you will need to define what you want  https://twitter.com/search?q=%23newhomes&src=typd and if that is the type of data you are after.
0

Featured Post

Free Tool: Subnet Calculator

The subnet calculator helps you design networks by taking an IP address and network mask and returning information such as network, broadcast address, and host range.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Introduction If you're like most people, you have occasionally made a typographical error when you're entering information into an online form.  And to your consternation, the browser remembers the error, and offers to autocomplete your future entr…
A phishing scam that claims a recipient’s credit card details have been “suspended” is the latest trend in spoof emails.
Google currently has a new report that is in beta and coming soon to Webmaster Tool accounts. This Micro Tutorial will highlight new features for Google Webmaster Tools.
This Micro Tutorial will demonstrate how to add subdomains to your content reports. This can be very importing in having a site with multiple subdomains.

789 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question