asked on

How do I curl or fopen final javascript source?

I am trying to extract data from a javascript webpage. Normally for a html page I can use cUrl or fopen and grab the source code then look for and match data I need, but with in those pages there are parts that are javascript and in those parts I can only see javascript code, not the final data like when veiwing from a web browser.

How can I use cUrl and or fopen to view and preg_match through the final Javascript code?

<table>
<tr>
    <td>
<script language="JavaScript" type="text/javascript">
<!--
ctxt_rk_bc = "EC690E";
ctxt_rk_cc = "FFFFFF";
ctxt_rk_lc = "0033FF";
ctxt_rk_tc = "333333";
ctxt_rk_uc = "666666";
// -->
</script>
<script language="JavaScript" src="http://www.sample.com/test.js">
</script>
 
</td>
  </tr>
<table>

Open in new window

onethreefour

It sounds like your problem is that some of the javascript code is 'hidden' in .js files which are called from 'script src' within the html page... You could use the php split function to parse the URL of the .js file out of the html page.
http://us2.php.net/manual/en/function.split.php

Then use cURL or fopen to pull down the .js file which you could then run your preg_match on.
http://us3.php.net/manual/en/function.preg-match.php

The initial split will be a little dirty, because you will need to split everything off of the front of the javascript src url, then you will need to split off everything after the url... Leaving you with just the URL to the .js file, which you then cURL and then preg_match...
Something like:
$stuff = split('<script language="JavaScript" src="', $wholeHTMLpage);
$stuff2 = split('">',$stuff);
$result = cURL($stuff2);
I would prefer to do this in perl, my php is a bit rusty and the above code is probably not exactly correct for php, but hopefully you get the point...

racmail2001

i can understand something else from your question.

i think that what you are trying to say is:

you whant to grab a page that is generated trough javascript/ajax/jquery and to grab the result of that

i don't think it's posible because javascript it's executed on the client side after all the content it's pulled of the server. with fopen or curl you grab the content of the server (or some of it - not the images, the css, js, etc) and than you shuld run the javascript .

but from php you can't do that

marcelof13

ASKER

hi Racmail2001,
There is got to be some way to do that. If a browser can do it, why cant some small amount of code?

ASKER CERTIFIED SOLUTION

dinu_sorin

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

marcelof13

ASKER

Come on guys, there is got to be a solution out there. Going to Delphi cant be the answer, can it?

SOLUTION

racmail2001

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

SOLUTION

b0lsc0tt

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

b0lsc0tt

You might look at http://j4p5.sourceforge.net/ . That project seems to be trying to make something for PHP like what you would need. Notice the limitations though and you should check the results. I am not aware of how this works or similar projects but there may be others. IMO they aren't something that will work for all cases though so it really will depend on exactly what you are working with and trying to get.
bol

marcelof13

ASKER

Thanks guys, so far none of the leads have helped me out.

b0lsc0tt

If the project I suggested won't work then I don't know of anything else for PHP. If you can use Java (i.e. JSP) then I believe there is an option you can try. Let me know if you want info on it.
If this has to be in PHP and do this then you could revisit this in a few months or a year. At that time the project I suggested or another may work for PHP.
bol

marcelof13

ASKER

Please post the information using Java or JSP, thanks.

b0lsc0tt

Java isn't my expertise, I have only done a little work in it, but from what I have read they do have access to a "Javascript engine." The best info on the topic I have found is at http://java.sun.com/javase/6/docs/technotes/guides/scripting/programmer_guide/index.html . I haven't even tested the code mentioned there, don't know enough about JSP to know if it can be used in that case, or know if that engine would run the Javascript you need. However it is at least a start if you can use Java and would seem to do what you need.
Let me know how it helps or if you have a question.
bol

b0lsc0tt

marcelof13,

Although the answer wasn't what you hoped I believe you did get an answer here. You were told the methods you were using would definitely not work and why. An option that might work in PHP was presented. Since that didn't work you were told PHP couldn't do it and given an alternative. Basically the answer was "you can't" and that can be a valid answer. If you disagree please comment and explain but I feel this was answered.

By the way since the details of the other language were posted at your request it seems odd that you appear to use them as an excuse to delete this. Just an observation. :) I really am sorry there wasn't something in PHP for this (yet) but this definitely wasn't an easy thing you are trying to do.

bol

marcelof13

ASKER

bol,
I hear you. Sorry I didn't realize the answer "you can't" was valid, I didn't really know how to end the question and also thought that if no answer was given or an answer that could not really help you then it was considered a not answered. It doesn't matter to me to give points to people who tried to help.

Thanks for your help, its very much appreciated.

b0lsc0tt

marcelof13,

Your welcome! Is the grade a mistake though? It is the lowest you can give. It would normally be used when the answer was incomplete and the experts did not participate well. Do you feel either is the case? If the grade was just an accident then let me know what you meant to award. From your comment it seems you agreed with what I said in my previous comment. The C grade definitely doesn't fit then. Otherwise please explain. If you do have a question about grading on EE or what they mean then let me know.

bol

marcelof13

ASKER

Bol,
I graded again with the way the answers helped me. I tried to explain in the comment text field. Sorry i didnt realize that also effected you guy. Yes you tried and again I thank you for your help.

Marcelo

marcelof13

ASKER

Was the solution complete: the answer is really, i dont know.

I dont know if the solution is accurate, I would have to try it and really dont have the time to test.