Solved

Quickest way to read in web site source code?

Posted on 2010-09-06
6
432 Views
Last Modified: 2012-05-10
I was wondering if there is a quicker way to read the source code of a website than the way I am doing it below?  Would it be quicker to use c# or c++ than vb.net?  I am using visual studio 2005.  Thanks



Dim webResponse3 As System.Net.HttpWebResponse = Nothing
Dim webRequest3 As System.Net.HttpWebRequest = System.Net.HttpWebRequest.Create("http:www.test.com")

webResponse3 = DirectCast(webRequest3.GetResponse(), System.Net.HttpWebResponse)
Dim srResp As System.IO.StreamReader
srResp = New System.IO.StreamReader(webResponse3.GetResponseStream())
Dim SOMESTRING As String
SOMESTRING = srResp.ReadToEnd
Dim webResponse3 As System.Net.HttpWebResponse = Nothing
Dim webRequest3 As System.Net.HttpWebRequest = System.Net.HttpWebRequest.Create("http:www.test.com")

webResponse3 = DirectCast(webRequest3.GetResponse(), System.Net.HttpWebResponse)
Dim srResp As System.IO.StreamReader
srResp = New System.IO.StreamReader(webResponse3.GetResponseStream())
Dim SOMESTRING As String
SOMESTRING = srResp.ReadToEnd

Open in new window

0
Comment
Question by:deer22
6 Comments
 
LVL 7

Expert Comment

by:jdavistx
ID: 33614217
Well, I'm not sure what you're ultimately wanting to do, but PHP has a pretty simple way of doing this.
http://www.php.net/file_get_contents
0
 
LVL 83

Expert Comment

by:Dave Baldwin
ID: 33614355
Here's a couple of alternate .NET ways to download from a website with both VB and C# code: http://blogs.techrepublic.com.com/programming-and-development/?p=695

PHP has several ways also including 'curl' if you need to emulate a browser and send all the headers.  The .NET article notes that some sites require the appropriate HTTP headers to access pages.
0
 
LVL 3

Author Comment

by:deer22
ID: 33614426
What I want to do is read in the websites html source code to a string and scan through the text using vb.net.

I'm wondering what is the quickest way to get the source code read from the site and into the string variable?
0
Master Your Team's Linux and Cloud Stack!

The average business loses $13.5M per year to ineffective training (per 1,000 employees). Keep ahead of the competition and combine in-person quality with online cost and flexibility by training with Linux Academy.

 
LVL 7

Expert Comment

by:Inteqam
ID: 33614897
0
 
LVL 7

Expert Comment

by:Inteqam
ID: 33614898
Imports System.Net
Imports System.IO
Public Class Form1
    Private Sub Button1_Click(ByVal sender As System.Object, _
    ByVal e As System.EventArgs) Handles Button1.Click
        Dim inStream As StreamReader
        Dim webRequest As WebRequest
        Dim webresponse As WebResponse
        webRequest = webRequest.Create(TextBox1.Text)
        webresponse = webRequest.GetResponse()
        inStream = New StreamReader(webresponse.GetResponseStream())
        TextBox2.Text = inStream.ReadToEnd()
    End Sub
End Class
0
 
LVL 16

Accepted Solution

by:
kris_per earned 500 total points
ID: 33616865

The way you are doing (using HttpWebRequest) is a quickest way to get the html source of a wbe page...the important part comes after that which is locating and reading the data you want...to locate and find the data in html following are some possible options:

1. use string type methods (like indexof/substring) - quickest but not a clean method OR
2. load html into XmlDocument and the use SelectSingleNode with xpath query to locate/get data OR
3. use external html parser classes like HTML Agility pack (which again uses xpath queries I think) => http://htmlagilitypack.codeplex.com/
0

Featured Post

Free Tool: Subnet Calculator

The subnet calculator helps you design networks by taking an IP address and network mask and returning information such as network, broadcast address, and host range.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Real-time is more about the business, not the technology. In day-to-day life, to make real-time decisions like buying or investing, business needs the latest information(e.g. Gold Rate/Stock Rate). Unlike traditional days, you need not wait for a fe…
If you need to start windows update installation remotely or as a scheduled task you will find this very helpful.
Microsoft Active Directory, the widely used IT infrastructure, is known for its high risk of credential theft. The best way to test your Active Directory’s vulnerabilities to pass-the-ticket, pass-the-hash, privilege escalation, and malware attacks …
A short tutorial showing how to set up an email signature in Outlook on the Web (previously known as OWA). For free email signatures designs, visit https://www.mail-signatures.com/articles/signature-templates/?sts=6651 If you want to manage em…

828 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question