Solved

Web Scrape Javascript Site

Posted on 2011-09-25
3
423 Views
Last Modified: 2012-09-27
I have never done web scraping before, but have a situation where it would help a ton.  I go to a site that runs a javascript to display data.  I use Chrome and right click the page, and view page source and completely understand the page and can see the data that I want to grab.  I want to do 2 things; play a sound or some notification when a certain number is reached and store data for analysis.  I have done research and found many tools like greasemonkey for firefox, AJAX,  irobotsoft, etc.  I am looking for guidance as to what would be the best way to do this.  I have no problem leaving the site open and logged in while the scraping would be done.
0
Comment
Question by:etechit
3 Comments
 
LVL 82

Expert Comment

by:leakim971
ID: 36601110
0
 
LVL 1

Author Comment

by:etechit
ID: 36602475
Thank you, but the site I am looking at is javascript and these articles don't refer to that at all
0
 
LVL 17

Accepted Solution

by:
selvol earned 500 total points
ID: 38357259
Im going to make it short but hopefully meaningful.....

The 1 thing I do not Understand is the site runs a "JAVASCRIPT" to display.......
That can be a couple of things...


I use and have used Metaproducts Offline Explorer for scraping for 10+ Years,
It can Login, scrape Parts of the page, ajax and many other features.
It is developed by One of the Best developers Code wise and Support wise I've seen.


Please don't think I am just tossing some cheap, lame down loader at you because I used it once.
I am not sure of your exact needs. But from what I read it looks like OE can assist.

Give it a try. If you have any doubts or questions ask me here.

Or go to the Metaproducts forum page.  And ask the actual developer. He usually responds fast.

Selvol



0

Featured Post

3 Use Cases for Connected Systems

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, testing some more, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Learn by example how to specify CSS selectors for Selenium WebDriver test automation software.
Using SQL Scripts we can save all the SQL queries as files that we use very frequently on our database later point of time. This is one of the feature present under SQL Workshop in Oracle Application Express.
Any person in technology especially those working for big companies should at least know about the basics of web accessibility. Believe it or not there are even laws in place that require businesses to provide such means for the disabled and aging p…
The is a quite short video tutorial. In this video, I'm going to show you how to create self-host WordPress blog with free hosting service.

920 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now