Advanced HttpWebRequest Issue

Hello,

High level overview, we are executing a httpwebrequest that is returning a 200 success message but no data.  Here are the details.

The URI is hitting a domain maintained by a third party vendor which does language translations

They use a simple proxy for that domain to point back at our code base

When we place the URI in a browser, the desired results are returned just fine.  We also notice from the browser http headers that the encoding is gzip.

Best buy is also using this same vendor's service for spanish translation, here is their url that can be used to test.  We get same results from this oen as we do ours. Can't use our URI yet cause it hits our test environment which is publicly accessible.

http://espanol.bestbuy.com/enes/

Lastly, our request again returns a success but no data. Content-length = -1  and the content-encoding = "" (Nothing) so it seems we aren't returning anything.

Can anyone hit this Best Buy url with our code and let us know what we are doing wrong?

PLEASE DO NOT POST A URL FOR ME TO LOOK AT!!  Unless you have a really good idea that it pertains to this specific issue. This is my second posting and the one before, everybody kept providing urls to the most basic http web request tutorials.  This code works for 99% of urls, just not the one(s) referenced.

Thanks in advance..
Dim httpRequest As HttpWebRequest = CType(HttpWebRequest.Create("http://espanol.bestbuy.com/enes/"), HttpWebRequest)

httpRequest.KeepAlive = True
httpRequest.Method = "GET"

            httpRequest.Headers.Add(HttpRequestHeader.Accept, "text/html,application/xhtml+xml,application/xml;")         httpRequest.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip,deflate")        httpRequest.Headers.Add(HttpRequestHeader.AcceptCharset, "ISO-8859-1,utf-8;")        httpRequest.Headers.Add(HttpRequestHeader.AcceptLanguage, "en-us,en;")

httpRequest.AllowAutoRedirect = True


Dim httpResponse As HttpWebResponse = CType(httpRequest.GetResponse(), HttpWebResponse)

Dim streamer As Stream = httpResponse.GetResponseStream()

If (httpResponse.ContentEncoding.ToLower().Contains("gzip")) Then
streamer = New System.IO.Compression.GZipStream(streamer, Compression.CompressionMode.Decompress)
ElseIf (httpResponse.ContentEncoding.ToLower().Contains("defalte")) Then
streamer = New System.IO.Compression.DeflateStream(streamer, Compression.CompressionMode.Decompress)
End If

Dim reader As New StreamReader(streamer, Encoding.Default)

body = reader.ReadToEnd()

Open in new window

LVL 4
kruegersteAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

PaulHewsCommented:
Some sites require a useragent before they respond with anything useful.
        Dim httpRequest As HttpWebRequest = CType(HttpWebRequest.Create("http://espanol.bestbuy.com/enes/"), HttpWebRequest)

        httpRequest.KeepAlive = True
        httpRequest.Method = "GET"
        httpRequest.Accept = "text/html,application/xhtml+xml,application/xml;"

        httpRequest.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip,deflate")
        httpRequest.Headers.Add(HttpRequestHeader.AcceptCharset, "ISO-8859-1,utf-8;")
        httpRequest.Headers.Add(HttpRequestHeader.AcceptLanguage, "en-us,en;")
        httpRequest.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US)"
        httpRequest.AllowAutoRedirect = True


        Dim httpResponse As HttpWebResponse = CType(httpRequest.GetResponse(), HttpWebResponse)

        Dim streamer As Stream = httpResponse.GetResponseStream()

        If (httpResponse.ContentEncoding.ToLower().Contains("gzip")) Then
            streamer = New System.IO.Compression.GZipStream(streamer, Compression.CompressionMode.Decompress)
        ElseIf (httpResponse.ContentEncoding.ToLower().Contains("defalte")) Then
            streamer = New System.IO.Compression.DeflateStream(streamer, Compression.CompressionMode.Decompress)
        End If

        Dim reader As New StreamReader(streamer, Encoding.Default)

        Dim body As String = reader.ReadToEnd()
        Debug.WriteLine(body)

Open in new window

0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
PaulHewsCommented:
Also make sure that you close/dispose of all stream objects you open.
0
PaulHewsCommented:
And you should change

ElseIf (httpResponse.ContentEncoding.ToLower().Contains("defalte"))

To

ElseIf (httpResponse.ContentEncoding.ToLower().Contains("deflate"))
0
Exploring SQL Server 2016: Fundamentals

Learn the fundamentals of Microsoft SQL Server, a relational database management system that stores and retrieves data when requested by other software applications.

kruegersteAuthor Commented:
Thanks for the suggestion on the userAgent, did you actually try it and get it to work?  I will try it now.

Thanks for pointing out typo too.  
0
PaulHewsCommented:
I did try it, and it returned content in the body at least.
0
kruegersteAuthor Commented:
Perfect!!!!!!!!!!!!!!!!
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
.NET Programming

From novice to tech pro — start learning today.