how to do javascript/ajax web scraping in php (data extraction / parsing)
Recently I've been hired to do various scrapings but I've come across an unusual website that appears to use some form of AJAX to list its inventory.
Anyway... here's the URL:
http://apps.dealerconnection.com/dealers/bradleyford/usedvehicles?lang=en
Sometimes if Javascript is used you can parse the javascript because it becomes static upon execution, but I believe this uses ajax, so I have no idea what to do at this point. The only thing that I can think to do is maybe try to find the true source of the information, which from my understanding, still has to be some kind of server side script like php or asp... right?
Can anyone find it? I'm lost...
Anyway... here's the URL:
http://apps.dealerconnection.com/dealers/bradleyford/usedvehicles?lang=en
Sometimes if Javascript is used you can parse the javascript because it becomes static upon execution, but I believe this uses ajax, so I have no idea what to do at this point. The only thing that I can think to do is maybe try to find the true source of the information, which from my understanding, still has to be some kind of server side script like php or asp... right?
Can anyone find it? I'm lost...
Steve Bink
AJAX could be using a database back-end, so you may not ever be able to access the source. A good starting point would be to talk to the webmaster to find out if they have an API you can use to access their information.
this is a link to the js they use for the AJAX: http://content.dealerconnection.com/vfs/apps/js/yui/connection/connection-min.js
here is an example on that page:
YAHOO.util.Connect.asyncRe
so yes it does talk to a server side script using AJAX. you will need to get admin access to the website to get acccess to those files
here is an example on that page:
YAHOO.util.Connect.asyncRe
so yes it does talk to a server side script using AJAX. you will need to get admin access to the website to get acccess to those files
ASKER
Tagit, if the ajax is posting to a backend script... it must be doing so somewhere in the javascript... correct? Is it possible for me to create a php script to post to the same script?
Yes i don't see why not but you would need to pass the same arguments and variables that the scripts is expecting. give it a try
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.