Solved

Web Scrape Javascript Site

Posted on 2011-09-25
3
425 Views
Last Modified: 2012-09-27
I have never done web scraping before, but have a situation where it would help a ton.  I go to a site that runs a javascript to display data.  I use Chrome and right click the page, and view page source and completely understand the page and can see the data that I want to grab.  I want to do 2 things; play a sound or some notification when a certain number is reached and store data for analysis.  I have done research and found many tools like greasemonkey for firefox, AJAX,  irobotsoft, etc.  I am looking for guidance as to what would be the best way to do this.  I have no problem leaving the site open and logged in while the scraping would be done.
0
Comment
Question by:etechit
3 Comments
 
LVL 82

Expert Comment

by:leakim971
ID: 36601110
0
 
LVL 1

Author Comment

by:etechit
ID: 36602475
Thank you, but the site I am looking at is javascript and these articles don't refer to that at all
0
 
LVL 17

Accepted Solution

by:
selvol earned 500 total points
ID: 38357259
Im going to make it short but hopefully meaningful.....

The 1 thing I do not Understand is the site runs a "JAVASCRIPT" to display.......
That can be a couple of things...


I use and have used Metaproducts Offline Explorer for scraping for 10+ Years,
It can Login, scrape Parts of the page, ajax and many other features.
It is developed by One of the Best developers Code wise and Support wise I've seen.


Please don't think I am just tossing some cheap, lame down loader at you because I used it once.
I am not sure of your exact needs. But from what I read it looks like OE can assist.

Give it a try. If you have any doubts or questions ask me here.

Or go to the Metaproducts forum page.  And ask the actual developer. He usually responds fast.

Selvol



0

Featured Post

Master Your Team's Linux and Cloud Stack

Come see why top tech companies like Mailchimp and Media Temple use Linux Academy to build their employee training programs.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
What .NET URL re-routing tool did I use? 2 60
remove background quote mark from widget 6 55
ebay seller html tags 2 28
Writing and Selling Web App Based on Google Sheets 2 48
Why do we like using grid based layouts in website design? Let's look at the live examples of websites and compare them to grid based WordPress themes.
Any business that wants to seriously grow needs to keep the needs and desires of an international audience of their websites in mind. Making a website friendly to international users isn’t prohibitively expensive and can provide an incredible return…
The viewer will learn how to count occurrences of each item in an array.
The is a quite short video tutorial. In this video, I'm going to show you how to create self-host WordPress blog with free hosting service.

825 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question