Link to home
Start Free TrialLog in
Avatar of DColin
DColinFlag for Thailand

asked on

Reading HTML using System.IO.StreamReader

Hi Experts,

I am reading into my application many pages of HTML so that I can retrieve (scrape) data from them. My problem is that the data I require lies a thousand lines of HTML into the page. Having to read through these unwanted lines of code each time a data scrape is made is slowing things down. Is it possible to make an HTML page request of the server starting at line 1000 for example?
Avatar of wmestrom
wmestrom
Flag of Netherlands image

You can try if the server supports the range option in the HTTP header. Then you can specify a byte offset. The HTTP request would look something like this:

GET /somepage HTTP/1.1
Host: www.xyz.org
Range: bytes=123456-
Accept: *.*, */*

Hope this will work for you.

Greets
Willem
Avatar of DColin

ASKER

Hi wmestrom:

Do you know how I can use your answer with my existing code? Thanks.
        Dim MyRequest As System.Net.HttpWebRequest
        Dim MyResponse As System.Net.HttpWebResponse
        Dim MyStream As System.IO.StreamReader
 
        MyRequest = System.Net.WebRequest.Create("http://www.abc.com")
        MyResponse = MyRequest.GetResponse()
        MyStream = New System.IO.StreamReader(MyResponse.GetResponseStream())

Open in new window

ASKER CERTIFIED SOLUTION
Avatar of wmestrom
wmestrom
Flag of Netherlands image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial