?
Solved

Pull Data from Websites

Posted on 2014-02-09
6
Medium Priority
?
229 Views
Last Modified: 2014-03-06
Hi All,

I am trying to pull feeds from websites, like for new  homes from both websites, twitter and social media.  Does anyone know an easy way to pull this type of data and check for updates every hour or so?  Also, would like to import into a database.....

Thanks in advance

Jackson
0
Comment
Question by:Jack_son_
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
6 Comments
 
LVL 53

Assisted Solution

by:Scott Fell, EE MVE
Scott Fell,  EE MVE earned 1200 total points
ID: 39846398
Yes, check their api documentation and terms of service first.

One api could be http://www.zillow.com/howto/api/PropertyDetailsAPIOverview.htm. For any api you use, be sure to note the TOS http://www.zillow.com/howto/api/APITerms.htm it will explain the usage data limits.  

If you are talking about your website crawling another website and extract data, that is called screen scraping and for the most part is not legal as it probably violates copy rights.  For any homes type of site that is generated by a MLS (Multiple Listing Service) from a Realtor®, that data is protected.

If you can point to a specific site you want to use and the data is open source for your use, we can help give you guidance but using an api from something like zillow is going to be the best way to go.

There is a way to use Realtor® data through services like http://ww.idxbroker.com
0
 
LVL 13

Assisted Solution

by:Praveen DM
Praveen DM earned 800 total points
ID: 39847431
I have come across an old process , XML feed pull and populating them in DB .I don't remember the complete process though.

But the XML feed update is the free and most used across sites.
0
 
LVL 13

Assisted Solution

by:Praveen DM
Praveen DM earned 800 total points
ID: 39847439
0
Back Up Your Microsoft Windows Server®

Back up all your Microsoft Windows Server – on-premises, in remote locations, in private and hybrid clouds. Your entire Windows Server will be backed up in one easy step with patented, block-level disk imaging. We achieve RTOs (recovery time objectives) as low as 15 seconds.

 
LVL 53

Assisted Solution

by:Scott Fell, EE MVE
Scott Fell,  EE MVE earned 1200 total points
ID: 39847500
servoadmin, your answers are about creating a feed.  That is not what Jack_son_ is asking about.
0
 

Author Comment

by:Jack_son_
ID: 39863606
i just want to download some data from public sites and allow it to update as needed; is there a tool that can accomplish this and put it in a database?
0
 
LVL 53

Accepted Solution

by:
Scott Fell,  EE MVE earned 1200 total points
ID: 39863747
>i just want to download some data from public sites....for new  homes.....

That type of data is going to be tightly controlled. You are going to need a 3rd party service like http://www.idxbroker.com/

Creating a twitter feed should not be too hard, but you will need to define what you want  https://twitter.com/search?q=%23newhomes&src=typd and if that is the type of data you are after.
0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article offers some helpful and general tips for safe browsing and online shopping. It offers simple and manageable procedures that help to ensure the safety of one's personal information and the security of any devices.
A phishing scam that claims a recipient’s credit card details have been “suspended” is the latest trend in spoof emails.
This Micro Tutorial will demonstrate how to add subdomains to your content reports. This can be very importing in having a site with multiple subdomains.
Shows how to create a shortcut to site-search Experts Exchange using Google in the Chrome browser. This eliminates the need to type out site:experts-exchange.com whenever you want to search the site. Launch the Search Engine Menu: In chrome, via you…
Suggested Courses

752 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question