Link to home
Start Free TrialLog in
Avatar of ankurdk
ankurdk

asked on

Read code from Inspect Element

Hi,

I want to get "complete" page source from webpage but the "view source" option shows only HTML.  The code I want to access is only viewed using "Inspect Element" in Chrome/Firefox. Is there anyway, in C# or VB.NET to get the contents of Inspect element of the page ?
Avatar of lenamtl
lenamtl
Flag of Canada image

Hi,
This won't be possible to see it because server side code won't be displayed in the code source using a browser, you can only see  HTML, CSS, Javascript
(like when we do right click view source page).

To see the code source you need to open the pages in code editor like Notepad++ or any other code editor..
Your problem is (most likely) that the bulk of the page is being rendered as the result of Javascript running. You would have to find a way to execute that same Javascript after you download the initial page. You may be able to use Internet Explorer interop for this, or maybe even the WebBrowser control.
Avatar of ankurdk
ankurdk

ASKER

Thanks for your replies.

I am using the webbrowser control to store the complete HTML but it is not having the "dynamic" source.  Any way to read html elements ?
I'm not sure to follow you ,

You can see the entire HTML code using right click Inspect or right click view source code using Chrome.

To see the entire page code you need the original page and a code editor.

If you do not have access to the original files you won't be able to see the complete code, plus server side code rely on additional page that have no HTML code ect.

So you need to request the entire original package & DB if you want to see everything.
When you View Source of a web page, you're seeing the HTML source as it was downloaded from the server. If you have some dynamic content that manipulates the DOM, such as loading data via AJAX for example, then View Source won't show that. There used to be a WebDeveloper plugin for FireFox that had an additional option called View Generated Source, and that would show the full source including the new dynamically created content.

A slight work-around that you might try. Open up your web-page and press Ctrl+A. This will select everything on the page. Now right-click on your page and you'll see a menu item called View Selection Srouce - that will show you the source of the page including any dynamic content.

I've only tried this in FireFox and I have no idea how to acheive that programatically.
Avatar of ankurdk

ASKER

@lenamtl thanks for your reply.  I just need to view the code that appears when you open a page in Mozilla Firefox and select all the  matter pressing CTRL+A then right click and "view selection source"  the matter so appears is the exact code that I am looking for as has right been suggested by @Chris Stanyon.  Only my challange is how do I get it in .net.  I am using a Visual Studio 2008 VB.NET application using HttpWebRequest and HttpWebResponse but they fail to read the matter generated "dynamically".


@Chris Stanyon yours is the perfect reply to the requirement that I am looking for.  Thanks for this.  But as you correctly mentioned my need is how do I get it in .NET.
Hi Ankurdk,

Did you give a try to WebBrowser Control?

Regards,
Chinmay.
Avatar of ankurdk

ASKER

Dear Chinmay,

Thanks for your suggestion.  Yes, I did try the WebBrowser control in Visual Studio 2010 also. but again, it won't parse the "dynamic" (hidden) code.
I am not aware at what point of time this hidden code is added to the page, For Web Browser Control to work properly - how about waiting for a couple of seconds and then try? and if you don't mind and can share the page I will be more than happy to look at it and provide a better solution.
Avatar of ankurdk

ASKER

@Chinmay Patel - I am reading the page on the WebBrowser1_DocumentCompleted event which is triggered when the page is completely loaded.  But only limited HTML is displayed in webbrowser control.  On the contrary, the same page when seen in Mozilla Firefox using "view selection source" renders the complete source.

sharing the code would be difficult but on similar lines, you can refer to any website which dynamically renders output based on search inputs say for eg. the DHL or Fedex websites accept the tracking number and render the complete page but the actual status of the waybill number is dynamically rendered after a couple of seconds.  The dynamic source can be viewed using "view selection source" feature of Mozilla Firefox.  This source code is different and complete from the one viewed using "view source" feature just as @Chris Stanyon has mentioned.

Thanks.
ASKER CERTIFIED SOLUTION
Avatar of Chinmay Patel
Chinmay Patel
Flag of India image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial