Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

Is scraping a website allowed?

Posted on 2010-08-22
6
Medium Priority
?
553 Views
Last Modified: 2013-12-14
Is it okay to scrape a website for data?

It seems to me that if the data is publically available and even the source code for the page is visible through the View Source option, that scraping the website is also allowed.  But I need to hear what others think about this first.

What could the website do to block scraping?  Legally?  Technically?  

Would they need to block so that View Source fails?  Is that even possible?

I'm curious...

Thanks,
newbieweb

0
Comment
Question by:newbieweb
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
  • 2
  • 2
6 Comments
 
LVL 11

Accepted Solution

by:
mattibutt earned 1000 total points
ID: 33495591
even if you scrap the website google cache the website and its available if is no longer coming from you. blocking view-source is not possible with html.
if you are using java applet or flash then it will not be visible there are some other applications. there are techniques to disable browser saving file locally but then you are suppose to program  this for every single browser exists.
the source of website if written in server side will never be visible on the browser for instance if you used java servlet, php, .net or ruby nothing will ever be avilable publicy unless someone hacks into the webserver i am still confuse what are you trying to acheive
0
 
LVL 2

Assisted Solution

by:ghemstrom
ghemstrom earned 1000 total points
ID: 33495607
The webpage with its supporting data is protected by the publisher's copyright, which implies that you could read it but not republish it without the publishers consent. Even the linking to his page should not be done without consent if it is not evident that it is allowed...
0
 

Author Comment

by:newbieweb
ID: 33495660
I am not trying to block my website from scraping, I am trying to find if it's allowed for me to scrape other websites.  Websites exist with data I want to import.

I wonder if that is allowed, and if it's possible just by reading the HTML.
0
Will your db performance match your db growth?

In Percona’s white paper “Performance at Scale: Keeping Your Database on Its Toes,” we take a high-level approach to what you need to think about when planning for database scalability.

 
LVL 11

Assisted Solution

by:mattibutt
mattibutt earned 1000 total points
ID: 33495668
you need author permission perior to using their content if they have forbidden the use of their content if not you can use it
0
 
LVL 2

Assisted Solution

by:ghemstrom
ghemstrom earned 1000 total points
ID: 33495756
I would say that you must have a permission if the permission is not explicitely given on the webpage.
0
 

Author Closing Comment

by:newbieweb
ID: 33495773
Thanks.
0

Featured Post

Plesk WordPress Toolkit

Plesk's WordPress Toolkit allows server administrators, resellers and customers to manage their WordPress instances, enabling a variety of development workflows for WordPress admins of all skill levels, from beginners to pros.

See why 2/3 of Plesk servers use it.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Make the most of your online learning experience.
Successful collaboration among team members is essential for the growth of your business. When employees work together on projects, share ideas and communicate effectively they get better results.
Internet Business Fax to Email Made Easy - With  eFax Corporate (http://www.enterprise.efax.com), you'll receive a dedicated online fax number, which is used the same way as a typical analog fax number. You'll receive secure faxes in your email, f…
In this video we outline the Physical Segments view of NetCrunch network monitor. By following this brief how-to video, you will be able to learn how NetCrunch visualizes your network, how granular is the information collected, as well as where to f…

722 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question