Solved

Quickest way to read in web site source code?

Posted on 2010-09-06
6
431 Views
Last Modified: 2012-05-10
I was wondering if there is a quicker way to read the source code of a website than the way I am doing it below?  Would it be quicker to use c# or c++ than vb.net?  I am using visual studio 2005.  Thanks



Dim webResponse3 As System.Net.HttpWebResponse = Nothing
Dim webRequest3 As System.Net.HttpWebRequest = System.Net.HttpWebRequest.Create("http:www.test.com")

webResponse3 = DirectCast(webRequest3.GetResponse(), System.Net.HttpWebResponse)
Dim srResp As System.IO.StreamReader
srResp = New System.IO.StreamReader(webResponse3.GetResponseStream())
Dim SOMESTRING As String
SOMESTRING = srResp.ReadToEnd
Dim webResponse3 As System.Net.HttpWebResponse = Nothing
Dim webRequest3 As System.Net.HttpWebRequest = System.Net.HttpWebRequest.Create("http:www.test.com")

webResponse3 = DirectCast(webRequest3.GetResponse(), System.Net.HttpWebResponse)
Dim srResp As System.IO.StreamReader
srResp = New System.IO.StreamReader(webResponse3.GetResponseStream())
Dim SOMESTRING As String
SOMESTRING = srResp.ReadToEnd

Open in new window

0
Comment
Question by:deer22
6 Comments
 
LVL 7

Expert Comment

by:jdavistx
ID: 33614217
Well, I'm not sure what you're ultimately wanting to do, but PHP has a pretty simple way of doing this.
http://www.php.net/file_get_contents
0
 
LVL 83

Expert Comment

by:Dave Baldwin
ID: 33614355
Here's a couple of alternate .NET ways to download from a website with both VB and C# code: http://blogs.techrepublic.com.com/programming-and-development/?p=695

PHP has several ways also including 'curl' if you need to emulate a browser and send all the headers.  The .NET article notes that some sites require the appropriate HTTP headers to access pages.
0
 
LVL 3

Author Comment

by:deer22
ID: 33614426
What I want to do is read in the websites html source code to a string and scan through the text using vb.net.

I'm wondering what is the quickest way to get the source code read from the site and into the string variable?
0
DevOps Toolchain Recommendations

Read this Gartner Research Note and discover how your IT organization can automate and optimize DevOps processes using a toolchain architecture.

 
LVL 7

Expert Comment

by:Inteqam
ID: 33614897
0
 
LVL 7

Expert Comment

by:Inteqam
ID: 33614898
Imports System.Net
Imports System.IO
Public Class Form1
    Private Sub Button1_Click(ByVal sender As System.Object, _
    ByVal e As System.EventArgs) Handles Button1.Click
        Dim inStream As StreamReader
        Dim webRequest As WebRequest
        Dim webresponse As WebResponse
        webRequest = webRequest.Create(TextBox1.Text)
        webresponse = webRequest.GetResponse()
        inStream = New StreamReader(webresponse.GetResponseStream())
        TextBox2.Text = inStream.ReadToEnd()
    End Sub
End Class
0
 
LVL 16

Accepted Solution

by:
kris_per earned 500 total points
ID: 33616865

The way you are doing (using HttpWebRequest) is a quickest way to get the html source of a wbe page...the important part comes after that which is locating and reading the data you want...to locate and find the data in html following are some possible options:

1. use string type methods (like indexof/substring) - quickest but not a clean method OR
2. load html into XmlDocument and the use SelectSingleNode with xpath query to locate/get data OR
3. use external html parser classes like HTML Agility pack (which again uses xpath queries I think) => http://htmlagilitypack.codeplex.com/
0

Featured Post

DevOps Toolchain Recommendations

Read this Gartner Research Note and discover how your IT organization can automate and optimize DevOps processes using a toolchain architecture.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Calculating holidays and working days is a function that is often needed yet it is not one found within the Framework. This article presents one approach to building a working-day calculator for use in .NET.
It was really hard time for me to get the understanding of Delegates in C#. I went through many websites and articles but I found them very clumsy. After going through those sites, I noted down the points in a easy way so here I am sharing that unde…
This Micro Tutorial will teach you how to censor certain areas of your screen. The example in this video will show a little boy's face being blurred. This will be demonstrated using Adobe Premiere Pro CS6.
This video shows how to quickly and easily add an email signature for all users on Exchange 2016. The resulting signature is applied on a server level by Exchange Online. The email signature template has been downloaded from: www.mail-signatures…

773 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question