Best Web scrapping software?

What is the best Web Scrapping software?  I would like to scrap the details  available (in tabular form) on the links (around 40 links) of the following website.

http://www.amfiindia.com/amfimembers.aspx

Could you please help me understand the best software?

Thanks,
--Anand
FTbridgeAsked:
Who is Participating?
 
Ray PaseurCommented:
This takes a while to run, so please be patient.  
http://www.laprbass.com/RAY_temp_ftbridge.php

You may want to normalize the address a little bit before you sort (but there are only 40+ data elements so it should be straightforward).

<?php // RAY_temp_ftbridge.php
error_reporting(E_ALL);
echo '<pre>';

// THE URLS AND GETTING THE HTML DOCUMENT
$bas = 'http://www.amfiindia.com/';
$url = $bas . 'amfimembers.aspx';
$htm = file_get_contents($url);

// A SIGNAL STRING TO DECLOP THE HTML
$sig = '<a href="amfiMembers.aspx?mfid=';

// BREAK AND MANIPULATE
$arr = explode($sig, $htm);
unset($arr[0]);
foreach ($arr as $key => $str)
{
    $new = $sig . $str;
    $sub = explode('"', $new);
    $arr[$key] = trim($bas . $sub[1]);
}

// ACTIVATE THIS TO SEE THE LIST OF URLS
// print_r($arr);

// ITERATE OVER THE LINKS
$objs = array();
foreach ($arr as $lnk)
{
    // CREATE AN OBJECT TO HOLD THIS DATA AND READ THE PAGE
    $obj = new StdClass;
    $htm = file_get_contents($lnk);

    // ISOLATE THE NAME
    $sub = explode('Name of the Mutual Fund</td>', $htm);
    $sub = explode('</td>', $sub[1]);
    $sub = strip_tags($sub[0]);
    $obj->nam = $sub;

    // ISOLATE THE ADDRESS
    $sub = explode('Address of AMC</td>', $htm);
    $sub = explode('</td>', $sub[1]);
    $sub = strip_tags($sub[0]);
    $obj->add = $sub;

    // ISOLATE THE PHONE
    $sub = explode('Telephone Number</td>', $htm);
    $sub = explode('</td>', $sub[1]);
    $sub = strip_tags($sub[0]);
    $obj->fon = $sub;

    // ADD THIS OBJECT TO OUR ARRAY
    $objs[] = $obj;
}

// SHOW THE ACQUIRED DATA IN THE ARRAY OF OBJECTS
print_r($objs);

Open in new window

HTH, ~Ray
0
 
Lukasz ChmielewskiCommented:
There is no best, it depend on what you want. If you're into programming, you can write you own scrapper, if not - you can take a look at this links:

http://www.poynter.org/how-tos/digital-strategies/e-media-tidbits/102589/how-to-scrape-websites-for-data-without-programming-skills/

http://blog.outwit.com/?p=55
0
 
Ray PaseurCommented:
The information in that web site is © Association of Mutual Funds in India (AMFI) - Copyright 2013.  You need to get their permission to copy, store or repurpose the content.  But that aside, if you can post the exact URLs of one or two of the pages you want to extract and post an example of the data you're trying to get I will be able to show you how to use a PHP script to get this information.
0
Cloud Class® Course: CompTIA Healthcare IT Tech

This course will help prep you to earn the CompTIA Healthcare IT Technician certification showing that you have the knowledge and skills needed to succeed in installing, managing, and troubleshooting IT systems in medical and clinical settings.

 
FTbridgeAuthor Commented:
Thank you Ray_Paseur and Roads_Roads.

Ray_Paseur,
Here is what I want to do: On the following link there are links for several mutual funds on this page.
http://www.amfiindia.com/amfimembers.aspx

If go to the above URL you will see following fund as the first link:
-  BOI AXA Investment Managers Private Limited
If you click on this text you will see following page
http://www.amfiindia.com/amfiMembers.aspx?mfid=46

I would like to download name of the MF, Address and phone number for all the funds listed on the first page.

I am not going to sell or repurpose the contents. I just want to sort these mutual funds by the address. Thank you for your help in advance.
0
 
FTbridgeAuthor Commented:
Ray,

This is excellent! Thank you very much for your help on this! You have saved my time!

Best regards,
--Anand
0
 
Ray PaseurCommented:
Thanks for the points and thanks for using EE, ~Ray
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.