Solved

How do I curl or fopen final javascript source?

Posted on 2009-07-14
20
599 Views
Last Modified: 2012-05-07
I am trying to extract data from a javascript webpage.  Normally for a html page I can use cUrl or fopen and grab the source code then look for and match data I need, but with in those pages there are parts that are javascript and in those parts I can only see javascript code, not the final data like when veiwing from a web browser.

How can I use cUrl and or fopen to view and preg_match through the final Javascript code?
<table>

<tr>

    <td>

<script language="JavaScript" type="text/javascript">

<!--

ctxt_rk_bc = "EC690E";

ctxt_rk_cc = "FFFFFF";

ctxt_rk_lc = "0033FF";

ctxt_rk_tc = "333333";

ctxt_rk_uc = "666666";

// -->

</script>

<script language="JavaScript" src="http://www.sample.com/test.js">

</script>
 

</td>

  </tr>

<table>

Open in new window

0
Comment
Question by:marcelof13
  • 7
  • 6
  • 2
  • +2
20 Comments
 
LVL 2

Expert Comment

by:onethreefour
ID: 24856650
It sounds like your problem is that some of the javascript code is 'hidden' in .js files which are called from 'script src' within the html page... You could use the php split function to parse the URL of the .js file out of the html page.
http://us2.php.net/manual/en/function.split.php

Then use cURL or fopen to pull down the .js file which you could then run your preg_match on.
http://us3.php.net/manual/en/function.preg-match.php

The initial split will be a little dirty, because you will need to split everything off of the front of the javascript src url, then you will need to split off everything after the url... Leaving you with just the URL to the .js file, which you then cURL and then preg_match...
Something like:
$stuff = split('<script language="JavaScript" src="', $wholeHTMLpage);
$stuff2 = split('">',$stuff);
$result = cURL($stuff2);
I would prefer to do this in perl, my php is a bit rusty and the above code is probably not exactly correct for php, but hopefully you get the point...
0
 
LVL 10

Expert Comment

by:racmail2001
ID: 24856873
i can understand something else from your question.

i think that what you are trying to say is:

you whant to grab a page that is generated trough javascript/ajax/jquery and to grab the result of that

i don't think it's posible because javascript it's executed on the client side after all the content it's pulled of the server. with fopen or curl you grab the content of the server (or some of it - not the images, the css, js, etc) and than you shuld run the javascript .

but from php you can't do that
0
 

Author Comment

by:marcelof13
ID: 24857730
hi Racmail2001,
                     There is got to be some way to do that.  If a browser can do it, why cant some small amount of code?

0
 
LVL 2

Accepted Solution

by:
dinu_sorin earned 167 total points
ID: 24857959
marcelof13, if you want to execute the js code, you must use other language. With some language you can use IE api to explore websites. (try delphi, is good) Happy Coding!
0
 

Author Comment

by:marcelof13
ID: 24858814
Come on guys, there is got to be a solution out there.  Going to Delphi cant be the answer, can it?
0
 
LVL 10

Assisted Solution

by:racmail2001
racmail2001 earned 167 total points
ID: 24858852
in php for sure is NOT
0
 
LVL 54

Assisted Solution

by:b0lsc0tt
b0lsc0tt earned 166 total points
ID: 24860883
A web server language (e.g. .NET, PHP) doesn't really offer the answer (as experts have said above).  The closet is Java although you might need some class?/component? to do it.  Also I don't know if the results would work as you want.  I believe when I have heard it discussed the "engine" it has used is the same as Mozilla's so that part should be good and give decent results.  Unfortunately Java as a web language is one that I know the least about so I don't have a lot more info.
The general problem is you don't just need a language because you want the Javascript run and interpreted as the browser would.  That is what makes this not as easy as you may have originally thought.  .NET does support the JScript language, which is basically IE's version of Javascript, so that could be something to try if you use .NET.  This won't be perfect at all though because the languages aren't exactly the same and so the code may fail.  Also the results might not be the same as if you had the Javascript in the browser since it is .NET running the code.  The eval() function does work in JScript though so that is a bonus.
You should figure out exactly what you need from the JS source though?  Also what is the code that makes it?  You might have an easier option if you just need part and can "read" it with your server language, even PHP, and gets its intent.  Basically you aren't running it but you are reading it to make the result.  Any server language could do that but it is definitely limiting.
Let me know if you have a question about this.
bol
0
 
LVL 54

Expert Comment

by:b0lsc0tt
ID: 24861036
You might look at http://j4p5.sourceforge.net/ .  That project seems to be trying to make something for PHP like what you would need.  Notice the limitations though and you should check the results.  I am not aware of how this works or similar projects but there may be others.  IMO they aren't something that will work for all cases though so it really will depend on exactly what you are working with and trying to get.
bol
0
Easy Project Management (No User Manual Required)

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 

Author Comment

by:marcelof13
ID: 25010835
Thanks guys, so far none of the leads have helped me out.  
0
 
LVL 54

Expert Comment

by:b0lsc0tt
ID: 25015418
If the project I suggested won't work then I don't know of anything else for PHP.  If you can use Java (i.e. JSP) then I believe there is an option you can try.  Let me know if you want info on it.
If this has to be in PHP and do this then you could revisit this in a few months or a year.  At that time the project I suggested or another may work for PHP.
bol
0
 

Author Comment

by:marcelof13
ID: 25108084
Please post the information using Java or JSP, thanks.
0
 
LVL 54

Expert Comment

by:b0lsc0tt
ID: 25118252
Java isn't my expertise, I have only done a little work in it, but from what I have read they do have access to a "Javascript engine."  The best info on the topic I have found is at http://java.sun.com/javase/6/docs/technotes/guides/scripting/programmer_guide/index.html .  I haven't even tested the code mentioned there, don't know enough about JSP to know if it can be used in that case, or know if that engine would run the Javascript you need.  However it is at least a start if you can use Java and would seem to do what you need.
Let me know how it helps or if you have a question.
bol
0
 
LVL 54

Expert Comment

by:b0lsc0tt
ID: 25369678
marcelof13,

Although the answer wasn't what you hoped I believe you did get an answer here.  You were told the methods you were using would definitely not work and why.  An option that might work in PHP was presented.  Since that didn't work you were told PHP couldn't do it and given an alternative.  Basically the answer was "you can't" and that can be a valid answer.  If you disagree please comment and explain but I feel this was answered.

By the way since the details of the other language were posted at your request it seems odd that you appear to use them as an excuse to delete this.  Just an observation. :)  I really am sorry there wasn't something in PHP for this (yet) but this definitely wasn't an easy thing you are trying to do.

bol
0
 

Author Comment

by:marcelof13
ID: 25370534
bol,
       I hear you.  Sorry I didn't realize the answer "you can't" was valid, I didn't really know how to end the question and also thought that if no answer was given or an answer that could not really help you then it was considered a not answered.  It doesn't matter to me to give points to people who tried to help.

Thanks for your help, its very much appreciated.

   
0
 
LVL 54

Expert Comment

by:b0lsc0tt
ID: 25370722
marcelof13,

Your welcome!  Is the grade a mistake though?  It is the lowest you can give.  It would normally be used when the answer was incomplete and the experts did not participate well.  Do you feel either is the case?  If the grade was just an accident then let me know what you meant to award.  From your comment it seems you agreed with what I said in my previous comment.  The C grade definitely doesn't fit then.  Otherwise please explain.  If you do have a question about grading on EE or what they mean then let me know.

bol
0
 

Author Comment

by:marcelof13
ID: 25371269
Bol,
     I graded again with the way the answers helped me.  I tried to explain in the comment text field.  Sorry i didnt realize that also effected you guy.  Yes you tried and again I thank you for your help.

Marcelo
0
 

Author Closing Comment

by:marcelof13
ID: 31603589
Was the solution complete:  the answer is really, i dont know.

I dont know if the solution is accurate, I would have to try it and really dont have the time to test.
0

Featured Post

Enabling OSINT in Activity Based Intelligence

Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

Join & Write a Comment

Suggested Solutions

Title # Comments Views Activity
spacing 5 29
How can I assign SimpleXMLElements to variables? 5 30
Phone Dialer 5 36
while loop in html mail format 5 32
I imagine that there are some, like me, who require a way of getting currency exchange rates for implementation in web project from time to time, so I thought I would share a solution that I have developed for this purpose. It turns out that Yaho…
This article will give core knowledge of JavaScript and will head in to your first JavaScript program. I am Durvesh Naik and I am here to deal with this series of JavaScript. I will teach you JavaScript in part wise , as its quite boring to read big…
The viewer will learn how to look for a specific file type in a local or remote server directory using PHP.
The viewer will learn the basics of jQuery including how to code hide show and toggles. Reference your jQuery libraries: (CODE) Include your new external js/jQuery file: (CODE) Write your first lines of code to setup your site for jQuery…

706 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

19 Experts available now in Live!

Get 1:1 Help Now