Error: "you must write contentlength bytes to the request stream before calling begin getresponse" on webrequest

i have some doubts that the amount of bytes you were writing to the io stream really sum up to 100507.

the WriteString and especially WriteFile in VB could have different defaults regarding line feeds, binary data, alignment or length attributes.

you may consider writing byte arrays with explicit length instead of strings or file contents.

string s = "YourString";
byte[] bytes = System.Text.Encoding.UTF8.GetBytes(s);

io.Write(bytes, 0, bytes.Length);

Select allOpen in new window

Sara

Hi and thanks for answer,

I'm actually doing this already like this:

Public Sub WriteString(ByVal stream As Stream, ByVal str As String)
            Dim postData As Byte() = System.Text.Encoding.ASCII.GetBytes(str)
            stream.Write(postData, 0, postData.Length)
        End Sub

Select allOpen in new window

And with the file:

 Public Sub WriteFile(ByVal stream As Stream, ByVal filename As String)
            Using readIn As FileStream = New FileStream(filename, FileMode.Open, FileAccess.Read, FileShare.ReadWrite)
                readIn.Seek(0, SeekOrigin.Begin)
                Dim fileData As Byte() = New Byte(BufferSize - 1) {}
                Dim bytes As Integer
                Dim x = readIn.Read(fileData, 0, BufferSize)
                bytes = x
                'Do While (bytes = x) > 0
                '    stream.Write(fileData, 0, bytes)
                '    x = readIn.Read(fileData, 0, BufferSize)
                'Loop

                'While ((bytes = readIn.Read(fileData, 0, BufferSize)) > 0)
                '    stream.Write(fileData, 0, bytes)
                'End While
                While (CSharpImpl.__Assign(bytes, readIn.Read(fileData, 0, BufferSize))) > 0
                    stream.Write(fileData, 0, bytes)
                End While
            End Using

Select allOpen in new window

Thanks for help!

Peter

Actually this is the whole class:

Imports Microsoft.VisualBasic

Imports System
Imports System.Collections.Generic
Imports System.Linq
Imports System.Text
Imports System.IO
Imports System.Collections
Imports System.Net

Namespace DemoFileUpload
    Public Class MultipartForm
        Private coFormFields As Hashtable
        Protected coRequest As HttpWebRequest
        Private coFileStream As Stream
        Private Shared CONTENT_DISP As String = "Content-Disposition: form-data; name="
        Public Property TransferHttpVersion As Version
        Public Property FileContentType As String

        Public Sub New(ByVal url As String)
            sURL = url
            coFormFields = New Hashtable()
            ResponseText = New StringBuilder()
            BufferSize = 1024 * 10
            BeginBoundary = "ou812--------------8c405ee4e38917c"
            TransferHttpVersion = HttpVersion.Version11
            FileContentType = "application/pdf"
        End Sub

        Private _BeginBoundary As String

        Public Property BeginBoundary As String
            Get
                Return _BeginBoundary
            End Get
            Set(ByVal value As String)
                _BeginBoundary = value
                ContentBoundary = "--" & BeginBoundary
                EndingBoundary = ContentBoundary & "--"
            End Set
        End Property

        Protected Property ContentBoundary As String
        Protected Property EndingBoundary As String
        Public Property ResponseText As StringBuilder
        Public Property sURL As String
        Public Property BufferSize As Integer

        Public Sub SetFilename(ByVal path As String)
            coFileStream = New System.IO.FileStream(path, FileMode.Create, FileAccess.Write)
        End Sub

        Public Sub SetField(ByVal key As String, ByVal str As String)
            coFormFields(key) = str
        End Sub

        Public Overridable Function GetStream() As Stream
            Dim stream As Stream

            If coFileStream Is Nothing Then
                stream = coRequest.GetRequestStream()
            Else
                stream = coFileStream
            End If

            Return stream
        End Function

        Public Overridable Sub GetResponse()
            If coFileStream Is Nothing Then
                Dim stream As Stream
                Dim response As WebResponse

                Try
                    response = coRequest.GetResponse()
                Catch web As WebException
                    response = web.Response
                End Try

                If response IsNot Nothing Then
                    stream = response.GetResponseStream()
                    Dim sr As StreamReader = New StreamReader(stream)
                    Dim str As String
                    ResponseText.Length = 0

                    For Each str In sr.ReadLine
                        ResponseText.Append(str)
                    Next

                    response.Close()
                Else
                    Throw New Exception("MultipartForm: Error retrieving server response")
                End If
            End If
        End Sub

        Public Sub SendFile(ByVal filename As String)
            coRequest = CType(WebRequest.Create(sURL), HttpWebRequest)
            coRequest.ProtocolVersion = TransferHttpVersion
            coRequest.Method = "POST"
            coRequest.ContentType = "multipart/form-data; boundary=" & BeginBoundary
            coRequest.Headers.Add("Cache-Control", "no-cache")
            coRequest.KeepAlive = True
            Dim strFields As String = GetFormfields()
            Dim strFileHdr As String = GetFileheader(filename)
            Dim strFileTlr As String = GetFiletrailer()
            Dim info As FileInfo = New FileInfo(filename)
            coRequest.ContentLength = strFields.Length + strFileHdr.Length + strFileTlr.Length + info.Length
            Dim io As System.IO.Stream
            io = GetStream()
            WriteString(io, strFields)
            WriteString(io, strFileHdr)
            Me.WriteFile(io, filename)
            WriteString(io, strFileTlr)
            GetResponse()
            io.Close()
            coRequest = Nothing
        End Sub

        Public Sub WriteString(ByVal stream As Stream, ByVal str As String)
            Dim postData As Byte() = System.Text.Encoding.ASCII.GetBytes(str)
            stream.Write(postData, 0, postData.Length)
        End Sub

        Public Function GetFormfields() As String
            Dim str As String = ""
            Dim myEnumerator As IDictionaryEnumerator = coFormFields.GetEnumerator()

            While myEnumerator.MoveNext()
                str += ContentBoundary & vbCrLf & CONTENT_DISP & """" & myEnumerator.Key & """" & vbCrLf & vbCrLf & myEnumerator.Value & vbCrLf
            End While

            Return str
        End Function

        Public Function GetFileheader(ByVal filename As String) As String
            Return ContentBoundary & vbCrLf & CONTENT_DISP & """file""; filename=""" & Path.GetFileName(filename) & """" & vbCrLf & "Content-type: " & FileContentType & vbCrLf & vbCrLf
        End Function

        Public Function GetFiletrailer() As String
            Return vbCrLf & EndingBoundary
        End Function

        Public Sub WriteFile(ByVal stream As Stream, ByVal filename As String)
            Using readIn As FileStream = New FileStream(filename, FileMode.Open, FileAccess.Read, FileShare.ReadWrite)
                readIn.Seek(0, SeekOrigin.Begin)
                Dim fileData As Byte() = New Byte(BufferSize - 1) {}
                Dim bytes As Integer
                Dim x = readIn.Read(fileData, 0, BufferSize)
                bytes = x
                'Do While (bytes = x) > 0
                '    stream.Write(fileData, 0, bytes)
                '    x = readIn.Read(fileData, 0, BufferSize)
                'Loop

                'While ((bytes = readIn.Read(fileData, 0, BufferSize)) > 0)
                '    stream.Write(fileData, 0, bytes)
                'End While
                While (CSharpImpl.__Assign(bytes, readIn.Read(fileData, 0, BufferSize))) > 0
                    stream.Write(fileData, 0, bytes)
                End While
            End Using
        End Sub

        Private Class CSharpImpl
            <Obsolete("Please refactor calling code to use normal Visual Basic assignment")>
            Shared Function __Assign(Of T)(ByRef target As T, value As T) As T
                target = value
                Return value
            End Function
        End Class
    End Class
End Namespace

Select allOpen in new window

And this is how I invoke it:

 If FileUpload1.UploadedFiles.Count > 0 Then
            For Each f As UploadedFile In FileUpload1.UploadedFiles
                Dim path As String = Server.MapPath("../storage/Files/Contracts/")
                Dim fileName As String = path & f.GetName
                f.SaveAs(path & f.GetName)

                ' FileUpload1.PostedFile.SaveAs(path & FileUpload1.FileName)
                Dim form As MultipartForm = New MultipartForm("https://theurl")
                form.SetField("access_token", "*****")
                form.SetField("recipient_email", "n****r@gmail.com")
                form.SetField("recipient_name", "Peter Nordberg")
                form.SendFile(fileName)
                Dim s As StringBuilder = form.ResponseText

                If s.ToString().ToLower().Contains("success") Then
                    lblSuccess.Visible = True
                Else
                    lblSuccess.Visible = True
                    lblSuccess.Text = "Error"
                    lblSuccess.ForeColor = Color.Red
                End If
            Next

        End If

Select allOpen in new window

Encoding from Windows UTF16 Unicode to Ascii might give differences in length. if String.Length attribute is used it is not the length in bytes but the length in characters, which is 2-bytes in .NET. if you concatenate with vbCrLf and write it to text file windows will write only one linefeed character. this could be a further difference between VB and C#. i don't know exactly why you are using a file as an intermediate storage. 100 k data could be handled with a byte array or an array of fixed-sized blocks, with each block is 1024 or 4096.

did you check whether postData.Length is equal to the String.Length which was used to calculate the ContentsLength?

anyway, compare the lengths between VB and C# code. they definitively should be same. and as far as i could understand the error, the length of all bytes written to the stream must be identical to the contents length you specified.

Sara

Hi,

I'm so sorry I've been inactive on this post for so long. I had to go abroad and had no possibility to follow up.

I still get this error when I execute the script in vb (all works well when I do the same in C#).

System.Net.ProtocolViolationException: You must write ContentLength bytes to the request stream before calling [Begin]GetResponse

Select allOpen in new window

I've counted the bytes when I execute and it's the same in the vb.net script and C# script. So how do I get bytes in the request strean in vb?

I've seen that the same error can sometimes come in C# but is solved with this script

using (dataStream = Webrequest.GetRequestStream())
{
   dataStream.Write(byteArray, 0, byteArray.Length);
}

Select allOpen in new window

instead of

dataStream = Webrequest.GetRequestStream();
dataStream.Write(byteArray, 0, byteArray.Length); 

Select allOpen in new window

But how to do it in vb?!

Thanks for help!

Peter

write the data to a second file both in vb and c#. then compare the files by using a hex editor (for example visual studio binary editor).

Sara

Hm, I'm not totally sure exactly how to do that. I've never done it before.

Can you step me through?

Peter

Hi,

Where and how in this code can I copy the stream to a file?

 Public Sub SendFile(ByVal filename As String)
            coRequest = CType(WebRequest.Create(sURL), HttpWebRequest)
            coRequest.ProtocolVersion = TransferHttpVersion
            coRequest.Method = "POST"
            coRequest.ContentType = "multipart/form-data; boundary=" & BeginBoundary
            coRequest.Headers.Add("Cache-Control", "no-cache")
            coRequest.KeepAlive = True
            Dim strFields As String = GetFormfields()
            Dim strFileHdr As String = GetFileheader(filename)
            Dim strFileTlr As String = GetFiletrailer()
            Dim info As FileInfo = New FileInfo(filename)
            coRequest.ContentLength = strFields.Length + strFileHdr.Length + strFileTlr.Length + info.Length
            Dim io As System.IO.Stream
            io = GetStream()
            WriteString(io, strFields)
            WriteString(io, strFileHdr)
            Me.WriteFile(io, filename)
            WriteString(io, strFileTlr)
            GetResponse()
            io.Close()
            coRequest = Nothing
        End Sub

Select allOpen in new window

Public Sub WriteString(ByVal stream As Stream, ByVal str As String)
            Dim postData As Byte() = System.Text.Encoding.ASCII.GetBytes(str)
            stream.Write(postData, 0, postData.Length)
        End Sub

Select allOpen in new window

 

 Public Sub WriteFile(ByVal stream As Stream, ByVal filename As String)
            Using readIn As FileStream = New FileStream(filename, FileMode.Open, FileAccess.Read, FileShare.ReadWrite)
                readIn.Seek(0, SeekOrigin.Begin)
                Dim fileData As Byte() = New Byte(BufferSize - 1) {}
                Dim bytes As Integer
                Dim x = readIn.Read(fileData, 0, BufferSize)
                bytes = x
                'Do While (bytes = x) > 0
                '    stream.Write(fileData, 0, bytes)
                '    x = readIn.Read(fileData, 0, BufferSize)
                'Loop

                'While ((bytes = readIn.Read(fileData, 0, BufferSize)) > 0)
                '    stream.Write(fileData, 0, bytes)
                'End While
                While (CSharpImpl.__Assign(bytes, readIn.Read(fileData, 0, BufferSize))) > 0
                    stream.Write(fileData, 0, bytes)
                End While
            End Using
        End Sub

Select allOpen in new window

Thanks for help!

Peter

unfortunately i am not a vb programmer ...

you may try to open a file stream additional to the system.io stream:

Const vbstream As String = "c:\temp\vbstream.bin"
Dim fs As New FileStream( vbstream, FileMode.OpenOrCreate, FileAccess.Write)

Select allOpen in new window

then repeat all write statements to the io stream by using 'fs' instead.

finally close the file by fs.Close() .

in c# it is like

string csstream = @"c:\temp\csstream.bin";
FileStream fs = new FileStream(csstream, FileMode.OpenOrCreate, FileAccess.Write);

Select allOpen in new window

note, in your code there are already file names and file streams but their usage is not clear.

i would recommend to not mix up the new files with the existing ones. simply duplicate all write statements by changing the io stream to the fs stream.

Sara

Hi Sara,

First of all, thanks so much for your help, I really appriciate it.

I've now succeeded to create a file with the streams in both languages. I can open them in VS binary editor. I'm not totally sure how to effectivly evaluate them since there is a lot of signs and data in them. What am I looking for?

Peter

Hi again,

I've attached two images showing the start of the two files. As you can see from about line 00000200 they are beginning to be different. But how do I change that and correct the things that is different?

Peter

 

looks like some decoding went wrong and instead of decoded text there is some warning included.

you can find the wrong statement by adding a separation number sequence before each log statement.

WriteString(io, strFields)
WriteString(fs,  "---01---")
WriteString(fs, strFields)
WriteString(fs,  "---02---")
WriteString(fs, strFileHdr)

Select allOpen in new window

same thing do in c#.

that way you can spot the wrong write statement and perhaps debug where the wrong decoded string comes from.

Sara

Ok,

The error seems to occur in the sub WriteFile. In C# it looks like this:

 public void WriteFile(Stream stream, string filename)
        {
            using (FileStream readIn = new FileStream(filename, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
            {
                readIn.Seek(0, SeekOrigin.Begin); // move to the start of the file
                byte[] fileData = new byte[BufferSize];
                int bytes;
                while ((bytes = readIn.Read(fileData, 0, BufferSize)) > 0)
                {
                    // read the file data and send a chunk at a time
                    stream.Write(fileData, 0, bytes);
                }
            }
        }

Select allOpen in new window

I've tried to convert it to this (and tried some different versions, but obviously none works correctly):

 Public Sub WriteFile(ByVal stream As Stream, ByVal filename As String)
            Using readIn As FileStream = New FileStream(filename, FileMode.Open, FileAccess.Read, FileShare.ReadWrite)
                readIn.Seek(0, SeekOrigin.Begin)
                Dim fileData As Byte() = New Byte(BufferSize - 1) {}
                Dim bytes As Integer
                Dim x = readIn.Read(fileData, 0, BufferSize)
                bytes = x
                'Do While (bytes = x) > 0
                '    stream.Write(fileData, 0, bytes)
                '    x = readIn.Read(fileData, 0, BufferSize)
                'Loop

                While ((bytes = readIn.Read(fileData, 0, BufferSize)) > 0)
                    stream.Write(fileData, 0, bytes)
                End While
                'While (CSharpImpl.__Assign(bytes, readIn.Read(fileData, 0, BufferSize))) > 0
                '    stream.Write(fileData, 0, bytes)
                'End While
            End Using
        End Sub

Select allOpen in new window

Any idea how to do this in a right way in vb?

Peter

the WriteFile reads bytes from a file where the file name (path) was passed to SendFile by argument.

check the file name whether it is the same for VB and C#.

if different path or name, check the contents of the files. use text editor. if that fails, use hex editor.

check file date whether those files were old or newly created. try to find out where those files were written. likely from a service or maybe from your program itself???

Sara

It's the exact same file but another directory on the same computor. Should that matter fro writing and sending the file correctly?

Peter

I see from your code that the file path was constructed by

"../storage/Files/Contracts/" and a file name.  the file name comes from fileUpload1.uploadedFiles

the ../ means it is a relative path, likely relative to the folder where your program resides.

it is a good chance that your c# programs resides at a different folder than your vb program.

try to use an absolute path.

Sara

is it a text file? check if it has exactly the same length. use a command window (cmd.exe) and the command

fc path1 path2

Select allOpen in new window

use \ instead of / if the file compare fails. and use double quotes if the paths contain spaces.

Sara

alternatively open the files with the hex editor even if they are text files.

Sara

It's a pdf file. Here you can see the exact path in vb when it gets the file:

Where should I change to absolute path?

While (CSharpImpl.__Assign(bytes, readIn.Read(fileData, 0, BufferSize))) > 0

who did write this function? is it yours?

look at the end of the while statement. there is no closing paranthesis. is that a typo?

Sara

No it's an automatic conversion from c# from a converter. I've tried to change it and make it work as you can see, but I think the error is in how it reads and writes the stream there.

I've also opened the files uploaded both in the vb and the c# project in hex editor. They are exactly the same, from what I can see.

who writes those files? do they come from a new upload or are they old?

if they are pdf files they are binary files. and you have to read them byte for byte both in VB and C#. in order to get the same result, you must not do any decoding while reading the bytes because you can't guarantee that VB and C# are using the same crypt libraries.

can you describe where those pdf's are coming from and whether they are encoded or not?

Sara

The whole procedure is like this:

I upload an existing pdf file from my harddrive.
It's now in one of the web projects folders.
Now I execute the SendFile sub that holds the webrequest object the writing of form data and the writing of file data from the uploaded file.
Then it's supposed to be sent to the url in the webrequest, but in vb I get an error in the response object.

Peter

the receiving url should get the pdf data unchanged, right? or is there a translation necessary?

can you post the C# code of the WriteFile?

Sara

Yes it should translate it unchanged.

 public void WriteFile(Stream stream, string filename)
        {
            using (FileStream readIn = new FileStream(filename, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
            {
                readIn.Seek(0, SeekOrigin.Begin); // move to the start of the file
                byte[] fileData = new byte[BufferSize];
                int bytes;
                while ((bytes = readIn.Read(fileData, 0, BufferSize)) > 0)
                {
                    // read the file data and send a chunk at a time
                    stream.Write(fileData, 0, bytes);
                }
            }
        }

Select allOpen in new window

Peter

in VB you would do like

        Using rdr As New BinaryReader(readIn)
            'Read BufferSize bytes from the file
            Dim bytes(BufferSize) As Byte
            Dim bytesRead As Integer = rdr.Read(bytes, 0, BufferSize)
        End Using

Select allOpen in new window

i see another inconsistancy:

in VB you pass the io stream by value. In C# objects are passed by reference (as far as i know). if that is true, the stream in VB is a copy of the stream used in the calling function and therefore might have different internal stream status. at least the following statement after WriteFile goes wrong:

WriteString(io, strFileTlr)

Select allOpen in new window

because the original io stream has a wrong file position (not the one the copy has at end of sub WriteFile.

it is well possible that if you pass the io stream by reference in VB as well, that the issue is fixed. nevertheless i would use the BinaryReader as shown.

Sara

Thanks,

I altered the vb sub like this now:

 Public Sub WriteFile(ByVal stream As Stream, ByVal filename As String)
            Using readIn As FileStream = New FileStream(filename, FileMode.Open, FileAccess.Read, FileShare.ReadWrite)
                readIn.Seek(0, SeekOrigin.Begin)
                Dim fileData As Byte() = New Byte(BufferSize - 1) {}
                'Dim bytes As Integer
                ' Dim x = readIn.Read(fileData, 0, BufferSize)
                ' bytes = x

                Using rdr As New BinaryReader(readIn)
                    'Read BufferSize bytes from the file
                    Dim bytes(BufferSize) As Byte
                    Dim bytesRead As Integer = rdr.Read(bytes, 0, BufferSize)
                End Using
                'Do While (bytes = x) > 0
                '    stream.Write(fileData, 0, bytes)
                '    x = readIn.Read(fileData, 0, BufferSize)
                'Loop

                'While ((bytes = readIn.Read(fileData, 0, BufferSize)) > 0)
                '    stream.Write(fileData, 0, bytes)
                'End While
                'While (CSharpImpl.__Assign(bytes, readIn.Read(fileData, 0, BufferSize))) > 0
                '    stream.Write(fileData, 0, bytes)
                'End While
            End Using
        End Sub

Select allOpen in new window

Still I get this irritating error again:

"System.Net.ProtocolViolationException: You must write ContentLength bytes to the request stream before calling [Begin]GetResponse" error when calling to the "BeginGetResponse" method of the web request.

Select allOpen in new window

Peter

Public Sub WriteFile(ByVal stream As Stream, ByVal filename As String)

Select allOpen in new window

this should be changed to

Public Sub WriteFile(ByRef stream As Stream, ByVal filename As String)

Select allOpen in new window

Sara

I've changed it but same error. Is there anyway I can communicate the whole code to you? It has some data that I don't want to show publicly. I understand it must me hard to test while only having fragments.

Peter

as told i have no vb experience and no environment.

so i only can help by advice.

i checked the last version of the WriteFile again. You used the BinaryReader but all the Write functions are commented. that means that WriteFile doesn't write anything to the stream.

the original code in c# has a nested loop where it reads from pdf chunks of fixed length (BufferSize). then this buffer was written to stream within the loop. then the loop continues and the next buffer was filled.

the same thing must be done in VB

                Using rdr As New BinaryReader(readIn)
                    Dim bytes(BufferSize) As Byte
                    Dim bytesRead As Integer = 0
                
                    While ( (bytesRead = rdr.Read(bytes, 0, BufferSize) ) > 0)
                           stream.Write(bytes, 0, bytesRead)
                    End While
                End Using

Select allOpen in new window

Sara

Ok, sounds logical. For some reason though it keeps jumping over the loop so it doesn't write to teh stream. I made a screen cast so that you can see:

Screen video

if the writing still doesn't work you may use a BinaryWriter in the calling Sub  and pass the BinaryWriter object by reference to the WriteString and WriteFile subs. then you don't need to pass the io stream at all.

Sara

Hi Sara,

Thanks for your help! You are a hero and so grateful for the time you took. I can now succeccfully upload the form and file data through vb.

for everyone interesten in the solution it can be found here: #a42745534