?
Solved

Quickest way to read in web site source code?

Posted on 2010-09-06
6
Medium Priority
?
435 Views
Last Modified: 2012-05-10
I was wondering if there is a quicker way to read the source code of a website than the way I am doing it below?  Would it be quicker to use c# or c++ than vb.net?  I am using visual studio 2005.  Thanks



Dim webResponse3 As System.Net.HttpWebResponse = Nothing
Dim webRequest3 As System.Net.HttpWebRequest = System.Net.HttpWebRequest.Create("http:www.test.com")

webResponse3 = DirectCast(webRequest3.GetResponse(), System.Net.HttpWebResponse)
Dim srResp As System.IO.StreamReader
srResp = New System.IO.StreamReader(webResponse3.GetResponseStream())
Dim SOMESTRING As String
SOMESTRING = srResp.ReadToEnd
Dim webResponse3 As System.Net.HttpWebResponse = Nothing
Dim webRequest3 As System.Net.HttpWebRequest = System.Net.HttpWebRequest.Create("http:www.test.com")

webResponse3 = DirectCast(webRequest3.GetResponse(), System.Net.HttpWebResponse)
Dim srResp As System.IO.StreamReader
srResp = New System.IO.StreamReader(webResponse3.GetResponseStream())
Dim SOMESTRING As String
SOMESTRING = srResp.ReadToEnd

Open in new window

0
Comment
Question by:deer22
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
6 Comments
 
LVL 7

Expert Comment

by:jdavistx
ID: 33614217
Well, I'm not sure what you're ultimately wanting to do, but PHP has a pretty simple way of doing this.
http://www.php.net/file_get_contents
0
 
LVL 84

Expert Comment

by:Dave Baldwin
ID: 33614355
Here's a couple of alternate .NET ways to download from a website with both VB and C# code: http://blogs.techrepublic.com.com/programming-and-development/?p=695

PHP has several ways also including 'curl' if you need to emulate a browser and send all the headers.  The .NET article notes that some sites require the appropriate HTTP headers to access pages.
0
 
LVL 3

Author Comment

by:deer22
ID: 33614426
What I want to do is read in the websites html source code to a string and scan through the text using vb.net.

I'm wondering what is the quickest way to get the source code read from the site and into the string variable?
0
 [eBook] Windows Nano Server

Download this FREE eBook and learn all you need to get started with Windows Nano Server, including deployment options, remote management
and troubleshooting tips and tricks

 
LVL 7

Expert Comment

by:Inteqam
ID: 33614897
0
 
LVL 7

Expert Comment

by:Inteqam
ID: 33614898
Imports System.Net
Imports System.IO
Public Class Form1
    Private Sub Button1_Click(ByVal sender As System.Object, _
    ByVal e As System.EventArgs) Handles Button1.Click
        Dim inStream As StreamReader
        Dim webRequest As WebRequest
        Dim webresponse As WebResponse
        webRequest = webRequest.Create(TextBox1.Text)
        webresponse = webRequest.GetResponse()
        inStream = New StreamReader(webresponse.GetResponseStream())
        TextBox2.Text = inStream.ReadToEnd()
    End Sub
End Class
0
 
LVL 16

Accepted Solution

by:
kris_per earned 2000 total points
ID: 33616865

The way you are doing (using HttpWebRequest) is a quickest way to get the html source of a wbe page...the important part comes after that which is locating and reading the data you want...to locate and find the data in html following are some possible options:

1. use string type methods (like indexof/substring) - quickest but not a clean method OR
2. load html into XmlDocument and the use SelectSingleNode with xpath query to locate/get data OR
3. use external html parser classes like HTML Agility pack (which again uses xpath queries I think) => http://htmlagilitypack.codeplex.com/
0

Featured Post

Free Backup Tool for VMware and Hyper-V

Restore full virtual machine or individual guest files from 19 common file systems directly from the backup file. Schedule VM backups with PowerShell scripts. Set desired time, lean back and let the script to notify you via email upon completion.  

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Exception Handling is in the core of any application that is able to dignify its name. In this article, I'll guide you through the process of writing a DRY (Don't Repeat Yourself) Exception Handling mechanism, using Aspect Oriented Programming.
Performance in games development is paramount: every microsecond counts to be able to do everything in less than 33ms (aiming at 16ms). C# foreach statement is one of the worst performance killers, and here I explain why.
Add bar graphs to Access queries using Unicode block characters. Graphs appear on every record in the color you want. Give life to numbers. Hopes this gives you ideas on visualizing your data in new ways ~ Create a calculated field in a query: …
This tutorial will teach you the special effect of super speed similar to the fictional character Wally West aka "The Flash" After Shake : http://www.videocopilot.net/presets/after_shake/ All lightning effects with instructions : http://www.mediaf…

764 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question