Using, how to send POST request with post data and save the binary response to a file?

I am trying to write a console script that makes a POST request to a web server with post data and saves the response from the server to a file.  The server response is a compressed tar archive.  I'm able to make the request using weblclient upload data, but I cannot figure out a way to save the binary response to a variable and eventually to a file.

I am using framework 1.1.

Here is a snippet of my code so far:

    ' Download a single binary file from a server and save it to a local folder
    Public Sub DownloadAndSaveFile(ByVal Url As String, ByVal Filename As String, ByVal User As String, ByVal Pass As String, ByVal Debug As Integer)

        ' The post data template
        Dim PostdataTemplate As String = _

        Dim PostdataArray As Byte() = Encoding.ASCII.GetBytes(Postdata)

        ' Create a new NetworkCredential object
        Dim NetworkCredential As New NetworkCredential(User, Pass)

            ' Create a new WebClient instance
            Dim myWebClient As New WebClient

            ' Set Preauthenticate property to true
            'myWebClient.PreAuthenticate = True

            ' Associate the NetworkCrbedential object with the 'WebRequest' object
            myWebClient.Credentials = NetworkCredential

            ' Add required HTTP headers to request
            myWebClient.Headers.Add("Accept", "*/*")
            myWebClient.Headers.Add("Content-Type", "application/x-www-form-urlencoded")

            ' UploadData method implicitly sets HTTP POST as the request method
            Dim responseArray As Byte() = myWebClient.UploadData(Url, PostdataArray)
            ' The response array as byte generates an exception!

        Catch Ex As Exception
            WriteLine("Error: " & Ex.Message)

        End Try
    End Sub

I get an exception when saving the response as a byte:

Error: The underlying connection was closed: The server committed an HTTP protocol violation.

If I change the server to respond with ascii text, the response is correctly saved.  I think the issue is that I'm trying to save binary data to a variable that only handles text.

Can someone provide an example of how to send a POST request with parameters and then save the binary file that the server responds with to a local file?

Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

This problem occurs when the .NET Framework detects that the server response does not comply with HTTP 1.1 RFC. This problem may occur when the response contains incorrect headers or incorrect header delimiters

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
this link will be useful
Bootstrap 4: Exploring New Features

Learn how to use and navigate the new features included in Bootstrap 4, the most popular HTML, CSS, and JavaScript framework for developing responsive, mobile-first websites.

jamesspoAuthor Commented:
Hi udhayakumard,

You are correct.  The server is responding with an HTTP header with no name or value.  There is a colon, space and then carriage return and line feed.

I found that I could fix this by adding the following to either the machine.config or to MY_SCRIPT.exe.config:

<httpWebRequest useUnsafeHeaderParsing="true" />
I'll award you the points regardless, but I'm wondering if there is a fix for this within the script I'm writing (that wouldn't require modifying an external config file or adding a new config file).  The reason I ask is that I am providing this script executable to a third party and don't want them to have to make any further changes beyond running the script.

Thank you
jamesspoAuthor Commented:
I should note that I know that the server inserting a header with no name or value is against RFC2616.  But I don't have the ability to modify the server's response in the production environment.

U just place the script in ur customers server
jamesspoAuthor Commented:
Currently, the only method I know of to instruct vb to ignore the illegal header from the server is to add useUnsafeHeaderParsing="true" to either the machine.config or MY_SCRIPT.exe.config.

I'd like to only provide the customer with the .exe file and not force them to include the .config file change.

Is there a way to include this configuration option in my executable so I only provide the customer with a single file?


i think u cant do that programmatically... Not sure
jamesspoAuthor Commented:
Ok... I'll give you the points and hope that someone has an idea of how to set the option within the executable.

Thanks for your help!
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Visual Basic.NET

From novice to tech pro — start learning today.