Solved

Quickest way to read in web site source code?

Posted on 2010-09-06
6
423 Views
Last Modified: 2012-05-10
I was wondering if there is a quicker way to read the source code of a website than the way I am doing it below?  Would it be quicker to use c# or c++ than vb.net?  I am using visual studio 2005.  Thanks



Dim webResponse3 As System.Net.HttpWebResponse = Nothing
Dim webRequest3 As System.Net.HttpWebRequest = System.Net.HttpWebRequest.Create("http:www.test.com")

webResponse3 = DirectCast(webRequest3.GetResponse(), System.Net.HttpWebResponse)
Dim srResp As System.IO.StreamReader
srResp = New System.IO.StreamReader(webResponse3.GetResponseStream())
Dim SOMESTRING As String
SOMESTRING = srResp.ReadToEnd
Dim webResponse3 As System.Net.HttpWebResponse = Nothing
Dim webRequest3 As System.Net.HttpWebRequest = System.Net.HttpWebRequest.Create("http:www.test.com")

webResponse3 = DirectCast(webRequest3.GetResponse(), System.Net.HttpWebResponse)
Dim srResp As System.IO.StreamReader
srResp = New System.IO.StreamReader(webResponse3.GetResponseStream())
Dim SOMESTRING As String
SOMESTRING = srResp.ReadToEnd

Open in new window

0
Comment
Question by:deer22
6 Comments
 
LVL 7

Expert Comment

by:jdavistx
ID: 33614217
Well, I'm not sure what you're ultimately wanting to do, but PHP has a pretty simple way of doing this.
http://www.php.net/file_get_contents
0
 
LVL 82

Expert Comment

by:Dave Baldwin
ID: 33614355
Here's a couple of alternate .NET ways to download from a website with both VB and C# code: http://blogs.techrepublic.com.com/programming-and-development/?p=695

PHP has several ways also including 'curl' if you need to emulate a browser and send all the headers.  The .NET article notes that some sites require the appropriate HTTP headers to access pages.
0
 
LVL 3

Author Comment

by:deer22
ID: 33614426
What I want to do is read in the websites html source code to a string and scan through the text using vb.net.

I'm wondering what is the quickest way to get the source code read from the site and into the string variable?
0
What Should I Do With This Threat Intelligence?

Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

 
LVL 7

Expert Comment

by:Inteqam
ID: 33614897
0
 
LVL 7

Expert Comment

by:Inteqam
ID: 33614898
Imports System.Net
Imports System.IO
Public Class Form1
    Private Sub Button1_Click(ByVal sender As System.Object, _
    ByVal e As System.EventArgs) Handles Button1.Click
        Dim inStream As StreamReader
        Dim webRequest As WebRequest
        Dim webresponse As WebResponse
        webRequest = webRequest.Create(TextBox1.Text)
        webresponse = webRequest.GetResponse()
        inStream = New StreamReader(webresponse.GetResponseStream())
        TextBox2.Text = inStream.ReadToEnd()
    End Sub
End Class
0
 
LVL 16

Accepted Solution

by:
kris_per earned 500 total points
ID: 33616865

The way you are doing (using HttpWebRequest) is a quickest way to get the html source of a wbe page...the important part comes after that which is locating and reading the data you want...to locate and find the data in html following are some possible options:

1. use string type methods (like indexof/substring) - quickest but not a clean method OR
2. load html into XmlDocument and the use SelectSingleNode with xpath query to locate/get data OR
3. use external html parser classes like HTML Agility pack (which again uses xpath queries I think) => http://htmlagilitypack.codeplex.com/
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Introduction Hi all and welcome to my first article on Experts Exchange. A while ago, someone asked me if i could do some tutorials on object oriented programming. I decided to do them on C#. Now you may ask me, why's that? Well, one of the re…
If you need to start windows update installation remotely or as a scheduled task you will find this very helpful.
In this seventh video of the Xpdf series, we discuss and demonstrate the PDFfonts utility, which lists all the fonts used in a PDF file. It does this via a command line interface, making it suitable for use in programs, scripts, batch files — any pl…
This video demonstrates how to create an example email signature rule for a department in a company using CodeTwo Exchange Rules. The signature will be inserted beneath users' latest emails in conversations and will be displayed in users' Sent Items…

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

14 Experts available now in Live!

Get 1:1 Help Now