Link to home
Start Free TrialLog in
Avatar of arthurh88
arthurh88

asked on

VB.NET: Get an external website

I am trying to get a page of an external website's HTML into a variable.

Here is the code
Public Shared Function GetPageText(ByVal url As String) As String
 
        Dim request As HttpWebRequest = WebRequest.Create(url)
         request.Timeout = 20000
             Using response As HttpWebResponse = request.GetResponse()
                   Using streamIP As Stream = response.GetResponseStream()
                              Using readerText As New StreamReader(streamIP)
                                                Dim htmlText As String = readerText.ReadToEnd()
                            Return htmlText
                End Using
            End Using
        End Using
    End Function



here is the error it returns:
The remote server returned an error: (406) Not Acceptable.  

The break occurs at this line:    Using response As HttpWebResponse = request.GetResponse()

Any ideas?  
ASKER CERTIFIED SOLUTION
Avatar of darshan_derasari
darshan_derasari
Flag of India image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of arthurh88
arthurh88

ASKER

hi darshan.  same error.  for example, i could not get it to work with this site:

http://www  <dot>  innerpathways  <dot>  ca
i got it to work by doing this:

        myWebRequest.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)"
needed to add this line after the get:


        myWebRequest.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)"