How do price comparison sites work

Posted on 2007-10-20
Last Modified: 2013-12-09
Could anyone shed some light on how a price comparison site works, as in, do they use bots to search out product information or is it more labor intensive. For example a site like

Question by:nathan1038
    LVL 70

    Expert Comment

    by:Jason C. Levine
    Hi nathan1038,

    They use bots to scrape the current price of the site either at the time of your request or every so often.  

    Although I'd kind of like to see the operation that did that all manually.  That would be one set of stressed out people.

    LVL 7

    Expert Comment

    by:wittyslogan (one of the biggest in UK) comtakes commission from the companies who get sales from the site.  Direct Line Insurance (who are one of the more competetive Insurance companies in the UK) do not get listed in Moneysupermarket because they won't pay commission and also because they have threatened with legal action. They feel that price comparison sites only compare on price without anyother criteria taken into account.
    For reference go to this link:

    Also when you book insurance on moneysupermarket you often then have to complete the transaction by phone which seems a little backward to me.  Why would you stick a middle man in when you don't need to?
    LVL 74

    Accepted Solution

    Actually, most of them don't use bots to grab information from other sites.  They only list merchants who have uploaded their catalogue to the site.  While this is generally done automatically whenever the merchant modifies something on their own site, there is a fair amount of analysis and tweaking of pennies in order for a merchant's popular products to appear at the top of the list.

    You can usually find out how a site works by clicking on their "information for merchants" link... such as:

    These links are always at the BOTTOM of the page rather than in the general navigation areas.


    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    How to run any project with ease

    Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
    - Combine task lists, docs, spreadsheets, and chat in one
    - View and edit from mobile/offline
    - Cut down on emails

    Uploading files to the web server has become common part of almost any kind of web application. People use different technologies to solve this, but regardless of the technology used, it is always useful to have some kind of progress indicator shown…
    Author Note: Since this E-E article was originally written, years ago, formal testing has come into common use in the world of PHP.  PHPUnit ( and similar technologies have enjoyed wide adoption, making it possib…
    Viewers will get an overview of the benefits and risks of using Bitcoin to accept payments. What Bitcoin is: Legality: Risks: Benefits: Which businesses are best suited?: Other things you should know: How to get started:
    This tutorial walks through the best practices in adding a local business to Google Maps including how to properly search for duplicates, marker placement, and inputing business details. Login to your Google Account, then search for "Google Mapmaker…

    760 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    7 Experts available now in Live!

    Get 1:1 Help Now