Solved

Pull Data from Websites

Posted on 2014-02-09
6
223 Views
Last Modified: 2014-03-06
Hi All,

I am trying to pull feeds from websites, like for new  homes from both websites, twitter and social media.  Does anyone know an easy way to pull this type of data and check for updates every hour or so?  Also, would like to import into a database.....

Thanks in advance

Jackson
0
Comment
Question by:Jack_son_
  • 3
  • 2
6 Comments
 
LVL 52

Assisted Solution

by:Scott Fell, EE MVE
Scott Fell,  EE MVE earned 300 total points
ID: 39846398
Yes, check their api documentation and terms of service first.

One api could be http://www.zillow.com/howto/api/PropertyDetailsAPIOverview.htm. For any api you use, be sure to note the TOS http://www.zillow.com/howto/api/APITerms.htm it will explain the usage data limits.  

If you are talking about your website crawling another website and extract data, that is called screen scraping and for the most part is not legal as it probably violates copy rights.  For any homes type of site that is generated by a MLS (Multiple Listing Service) from a Realtor®, that data is protected.

If you can point to a specific site you want to use and the data is open source for your use, we can help give you guidance but using an api from something like zillow is going to be the best way to go.

There is a way to use Realtor® data through services like http://ww.idxbroker.com
0
 
LVL 13

Assisted Solution

by:Praveen DM
Praveen DM earned 200 total points
ID: 39847431
I have come across an old process , XML feed pull and populating them in DB .I don't remember the complete process though.

But the XML feed update is the free and most used across sites.
0
 
LVL 13

Assisted Solution

by:Praveen DM
Praveen DM earned 200 total points
ID: 39847439
0
VMware Disaster Recovery and Data Protection

In this expert guide, you’ll learn about the components of a Modern Data Center. You will use cases for the value-added capabilities of Veeam®, including combining backup and replication for VMware disaster recovery and using replication for data center migration.

 
LVL 52

Assisted Solution

by:Scott Fell, EE MVE
Scott Fell,  EE MVE earned 300 total points
ID: 39847500
servoadmin, your answers are about creating a feed.  That is not what Jack_son_ is asking about.
0
 

Author Comment

by:Jack_son_
ID: 39863606
i just want to download some data from public sites and allow it to update as needed; is there a tool that can accomplish this and put it in a database?
0
 
LVL 52

Accepted Solution

by:
Scott Fell,  EE MVE earned 300 total points
ID: 39863747
>i just want to download some data from public sites....for new  homes.....

That type of data is going to be tightly controlled. You are going to need a 3rd party service like http://www.idxbroker.com/

Creating a twitter feed should not be too hard, but you will need to define what you want  https://twitter.com/search?q=%23newhomes&src=typd and if that is the type of data you are after.
0

Featured Post

U.S. Department of Agriculture and Acronis Access

With the new era of mobile computing, smartphones and tablets, wireless communications and cloud services, the USDA sought to take advantage of a mobilized workforce and the blurring lines between personal and corporate computing resources.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

If your site has a few sections that need to be secure when data is transmitted between the server and local computer, such as a /order/ section for ordering or /customer/ which contains customer data, etc it would of course be recommended to secure…
A phishing scam that claims a recipient’s credit card details have been “suspended” is the latest trend in spoof emails.
Shows how to create a shortcut to site-search Experts Exchange using Google in the Chrome browser. This eliminates the need to type out site:experts-exchange.com whenever you want to search the site. Launch the Search Engine Menu: In chrome, via you…
How to create a custom search shortcut to site-search Experts Exchange using Google in the Firefox browser. This eliminates the need to type out site:experts-exchange.com whenever you want to search the site. Launch your Bookmark Menu: Press 'Ctrl +…

803 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question