What is the fastest way to scrape data from the web?

Posted on 2006-04-24
Last Modified: 2008-03-03
I run a small online bookstore and I scrape prices from sites like Amazon to make sure my prices are competitive. I use an Excel sheet and download the data direct into the Excel sheet. However it seems to me this process is very slow and somewhat unreliable (due to Excel "freezes"). Can somebody suggest a faster, mor reliable way to scrape data from the web into a database?
Question by:bpfsr
    LVL 7

    Expert Comment

    If you are looking for comparing prices from Amazon, I would suggest you to use the api provided by them to get the prices. Can you please give me more details on what you mean by downloading data to Excel sheet?
    LVL 3

    Assisted Solution

    Check this tool out, it accesses the Amazon web services for information:


    To install it, you'll need the .Net Framework 2.0:
    LVL 3

    Accepted Solution

    Alternatively, if you can code or are willing to hire someone who can you can have your own custom solution using the Amazon E-Commerce Service:

    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    Training Course: Android App Development

    This course will involve creating widgets, customize list view, grid view, spinners, etc. Creating applications using audio, video, and SQLite database. Ending with publication on Google Play.

    The first time you look at a web page and its source code, you are probably a little intimidated by the use of symbols and jargon that really looks foreign to you. You might not even know where to start to begin learning what it all means. That’…
    Foreword (July, 2015) Since I first wrote this article, years ago, a great many more people have begun using the internet.  They are coming online from every part of the globe, learning, reading, shopping and spending money at an ever-increasing ra…
    The viewer will learn how to dynamically set the form action using jQuery.
    The viewer will learn how to count occurrences of each item in an array.

    761 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    7 Experts available now in Live!

    Get 1:1 Help Now