Solved

Web Scrape Javascript Site

Posted on 2011-09-25
3
430 Views
Last Modified: 2012-09-27
I have never done web scraping before, but have a situation where it would help a ton.  I go to a site that runs a javascript to display data.  I use Chrome and right click the page, and view page source and completely understand the page and can see the data that I want to grab.  I want to do 2 things; play a sound or some notification when a certain number is reached and store data for analysis.  I have done research and found many tools like greasemonkey for firefox, AJAX,  irobotsoft, etc.  I am looking for guidance as to what would be the best way to do this.  I have no problem leaving the site open and logged in while the scraping would be done.
0
Comment
Question by:etechit
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
3 Comments
 
LVL 1

Author Comment

by:etechit
ID: 36602475
Thank you, but the site I am looking at is javascript and these articles don't refer to that at all
0
 
LVL 17

Accepted Solution

by:
selvol earned 500 total points
ID: 38357259
Im going to make it short but hopefully meaningful.....

The 1 thing I do not Understand is the site runs a "JAVASCRIPT" to display.......
That can be a couple of things...


I use and have used Metaproducts Offline Explorer for scraping for 10+ Years,
It can Login, scrape Parts of the page, ajax and many other features.
It is developed by One of the Best developers Code wise and Support wise I've seen.


Please don't think I am just tossing some cheap, lame down loader at you because I used it once.
I am not sure of your exact needs. But from what I read it looks like OE can assist.

Give it a try. If you have any doubts or questions ask me here.

Or go to the Metaproducts forum page.  And ask the actual developer. He usually responds fast.

Selvol



0

Featured Post

How Do You Stack Up Against Your Peers?

With today’s modern enterprise so dependent on digital infrastructures, the impact of major incidents has increased dramatically. Grab the report now to gain insight into how your organization ranks against your peers and learn best-in-class strategies to resolve incidents.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article discusses how to implement server side field validation and display customized error messages to the client.
Australian government abolished Visa 457 earlier this April and this article describes how this decision might affect Australian IT scene and IT experts.
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
The is a quite short video tutorial. In this video, I'm going to show you how to create self-host WordPress blog with free hosting service.

688 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question