html scrape problem

Hi everyone,

I am using asp.net with C#.

I have a page with a button. The button is NOT server-side. When clicking the button, an iframe on the page gets its src pointed (through a javascript function) to a page on a different domain which has a different querystring every time.

for example www.xyz.com/abc.html?v=123. Next time it can be ?v=456

This page has a div with a set id of "div1". Inside the div is an <a> whose href I must obtain.

Is there a way to scrape the page through javascript? Failing that, can I somehow have my C# do it after the iframe has been loaded?

TIA
Feivi99Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

hehdaddyCommented:
Hi there. You can easily do this using jQuery. If you don't know about jQuery, you can visit their site at http://www.jquery.com.

Essentially, all you need to do is download the jQuery .js file, place it in your project and put a reference to it in your page.

Then, in JavaScript you can access the iframe like this:

$('#iframeID').contents().find('#someID');

Does this make sense? jQuery is fully endorsed by Microsoft and will be included with all versions of Visual Studio in the future. Give it a try!
0
BadotzCommented:
Using jQuery to retrieve the content of an <iframe> is like using C4 to light a cigar.

Try this:


var ifr = document.getElementById('iframeId').contentWindow.document.body.innerHTML;
 
// Now you can parse the HTML for the element you need.

Open in new window

0
BadotzCommented:
Also, visit this link for a complete working example of RPC using an <iframe>.

The various browsers provide access to the content of the <iframe> in different ways; this is discussed in the article.

Here's the link:

http://developer.apple.com/internet/webcontent/iframe.html
0
Big Business Goals? Which KPIs Will Help You

The most successful MSPs rely on metrics – known as key performance indicators (KPIs) – for making informed decisions that help their businesses thrive, rather than just survive. This eBook provides an overview of the most important KPIs used by top MSPs.

Feivi99Author Commented:
hehdaddy,

I tried this:

var ifr = document.getElementById("myiframe");
var ddiv=$(ifr).contents().find('mydivid');
var val = ddiv.value;  //also tried ddiv.innerHTML
var tx = document.getElementById("Text1");
tx.value = val;

as a result I got "undefined" in my textbox.

Badotz, I tried it, but the textbox came up empty.

Are you both sure that Jquery also allows what we're trying to do cross-site?
0
Feivi99Author Commented:
Badotz,

I went over that article. Over there it only works when one has access to both pages, which isn't the case here.
0
hehdaddyCommented:
Hi there. I steered you in the wrong direction. Going cross-site causes security errors.

Aside, your jQuery was a bit off. I know that doesn't help now, but here it is nonetheless.

$('#myiframe').contents().find('mydivid').html();
0
williamcampbellCommented:
Maybe you would like to explore this option

http://geekswithblogs.net/ranganh/archive/2005/04/25/37635.aspx


0
Feivi99Author Commented:
hehdaddy,

I tried your code with a same-domain iframe, but I got no result. When I left off the () after the .html, I got

function (E) {     return E === g ? this[0] ? this[0].innerHTML.replace(/ jQuery\d+="(?:\d+|null)"/g, "") : null : this.empty().append(E); }

William,

That article just explains how to dynamically set teh src of an iframe. Not really the problem here...
0
williamcampbellCommented:
I was going to suggest as a follow up to handle the pre-render event on the HtmlControl and parse out the source there... looks like you want a Javascript Solution so no worries
0
Feivi99Author Commented:
alright, I ended up figuring out how to do it. I used a tiny iframe which points to another page on my domain which scrapes the needed page. Once that info is on that page, I could access it with a simple
document.getelementybyid("myiframe").contentwindow.document.getelementbyid("abc").etc.
0
BadotzCommented:
It seems to me that you were pointed in the right direction by several experts, leading you finally to the point where "alright, I ended up figuring out how to do it."

Give credit where credit is due.
0
Feivi99Author Commented:
I'm not sure what you mean. The solutions given ONLY worked for an iframe that points to a page on the SAME domain. I specifically wrote in the original question that I need it for a DIFFERENT domain, which was the entire point of the question! Referring to an element in an iframe pointing to the same domain doesn't need jquery AT ALL, as I demonstrated in my solution.
0
BadotzCommented:
So show us how you did it.
0
Feivi99Author Commented:
??

Did you just complain about my closing the post without reading what I wrote? I POSTED how I did it...???
0
Feivi99Author Commented:
I put an iframe in my page which has display:none. the src of of the iframe points to www.mydomain.com/scrape.aspx. On that page, I used the code found at
http://www.csharpfriends.com/Articles/getTip.aspx?articleID=210. This scrapes the page I need and makes the elements on that page part of the scrape.aspx DOM. Being that scrape.aspx is in my domain, I can now access the elements I need using regular HTML, i.e. with

document.getelementybyid("myiframe").contentwindow.document.getelementbyid("abc")
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
JavaScript

From novice to tech pro — start learning today.