Solved

Pull Data from Websites

Posted on 2014-02-09
6
220 Views
Last Modified: 2014-03-06
Hi All,

I am trying to pull feeds from websites, like for new  homes from both websites, twitter and social media.  Does anyone know an easy way to pull this type of data and check for updates every hour or so?  Also, would like to import into a database.....

Thanks in advance

Jackson
0
Comment
Question by:Jack_son_
  • 3
  • 2
6 Comments
 
LVL 52

Assisted Solution

by:Scott Fell, EE MVE
Scott Fell,  EE MVE earned 300 total points
ID: 39846398
Yes, check their api documentation and terms of service first.

One api could be http://www.zillow.com/howto/api/PropertyDetailsAPIOverview.htm. For any api you use, be sure to note the TOS http://www.zillow.com/howto/api/APITerms.htm it will explain the usage data limits.  

If you are talking about your website crawling another website and extract data, that is called screen scraping and for the most part is not legal as it probably violates copy rights.  For any homes type of site that is generated by a MLS (Multiple Listing Service) from a Realtor®, that data is protected.

If you can point to a specific site you want to use and the data is open source for your use, we can help give you guidance but using an api from something like zillow is going to be the best way to go.

There is a way to use Realtor® data through services like http://ww.idxbroker.com
0
 
LVL 13

Assisted Solution

by:servoadmin
servoadmin earned 200 total points
ID: 39847431
I have come across an old process , XML feed pull and populating them in DB .I don't remember the complete process though.

But the XML feed update is the free and most used across sites.
0
 
LVL 13

Assisted Solution

by:servoadmin
servoadmin earned 200 total points
ID: 39847439
0
What Security Threats Are You Missing?

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

 
LVL 52

Assisted Solution

by:Scott Fell, EE MVE
Scott Fell,  EE MVE earned 300 total points
ID: 39847500
servoadmin, your answers are about creating a feed.  That is not what Jack_son_ is asking about.
0
 

Author Comment

by:Jack_son_
ID: 39863606
i just want to download some data from public sites and allow it to update as needed; is there a tool that can accomplish this and put it in a database?
0
 
LVL 52

Accepted Solution

by:
Scott Fell,  EE MVE earned 300 total points
ID: 39863747
>i just want to download some data from public sites....for new  homes.....

That type of data is going to be tightly controlled. You are going to need a 3rd party service like http://www.idxbroker.com/

Creating a twitter feed should not be too hard, but you will need to define what you want  https://twitter.com/search?q=%23newhomes&src=typd and if that is the type of data you are after.
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

I annotated my article on ransomware somewhat extensively, but I keep adding new references and wanted to put a link to the reference library.  Despite all the reference tools I have on hand, it was not easy to find a way to do this easily. I finall…
Several part series to implement Internet Explorer 11 Enterprise Mode
Google currently has a new report that is in beta and coming soon to Webmaster Tool accounts. This Micro Tutorial will highlight new features for Google Webmaster Tools.
This Micro Tutorial will demonstrate how to add subdomains to your content reports. This can be very importing in having a site with multiple subdomains.

746 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now