Solved

Posting multipart/form-data as if the content was an uploaded file

Posted on 2004-10-15
10
1,046 Views
Last Modified: 2010-05-18
I am trying to post data to a web site as if the data was an uploaded file, here is my code:

Dim boundary as String = "---xxx"
Dim postData as String
postData += boundary  & Chr(10)
postData += "Content-Disposition: form-data; name=""upfile""; filename=""c:\xyz.xyz""" & Chr(10)
postData += "Content-Type: text/plain" & Chr(10)
postData += Chr(10)
postData += "file line 1" & Chr(10)
postData += "file line 2" & Chr(10)
postData += "file line 3" & Chr(10)
postData += boundary & Chr(10)
postData += "Content-Disposition: form-data; name=""submit""" & Chr(10)
postData += Chr(10)
ostData += "Upload()" & Chr(10)
postData += boundary & "--"

Dim req As Net.HttpWebRequest = System.Net.WebRequest.Create(url)
req.Method = "POST"
req.Headers.Add("Cookie", cookie)
req.ContentType = ""multipart/form-data; boundary=" & boundary
Dim postBuff() As Byte = System.Text.Encoding.GetEncoding(1252).GetBytes(postData)
req.ContentLength = postBuff.Length
Dim reqStream As System.IO.Stream = req.GetRequestStream()
reqStream.Write(postBuff, 0, postBuff.Length)
reqStream.Close()
Dim res As System.Net.HttpWebResponse = req.GetResponse()
Dim rdr As New System.Io.StreamReader(res.GetResponseStream(), System.Text.Encoding.ASCII)
Dim retVal as string = rdr.ReadToEnd()
res.Close()

However, this does not work!! Please help me, I am stuck.

Thanks, more points can easily be made available for a swift answer, I might have to post another question to do this as I can't see how I increase points right now
0
Comment
Question by:duncanlatimer
  • 5
  • 4
10 Comments
 
LVL 6

Expert Comment

by:der_jth
ID: 12329389
There could be various things wrong; it's rather hard to help since you're not telling _how_ it doesn't work. At least encoding issues are possible culprits; that 1252 is somewhat suspicious (though probably harmless), as is ASCII for the file contents (quite likely less harmless). For some quick tips, try reading the following blog entry: <http://blogs.ronaco.com/Blogs/rwilson/articles/174.aspx>. It is in C#, but it could help you to squash some issues.

Let's get back to this after you've posted some details on _how_ it doesn't work.
0
 

Author Comment

by:duncanlatimer
ID: 12332997
Sorry for not being able to be more specific here, the web site accepts the "file" with no error message and returns the same response as I would get if I had posted a "real file", however it does not process the information in the file, so I have to suppose that in some way the file is not processible. The format of the "file" content posted is correct, so I assume the way I post it isn't. I suppose my question is does my code emulate what happens when a file is posted and are there any other tricks I should try, I will look at the encoding, thanks.

As it is not my web site I am posting to I have absolutely no idea what is happening on the server side and obviously these people are expecting a "real file" so are unlikely to be very helpful if I ask them.
0
 

Author Comment

by:duncanlatimer
ID: 12333585
I have tried this with CRLF and well as just LF, still didn't work, also I use the very same post code to logon to the site, so this is known to work fine, I have changed the encoding to UTF8 and this works fine for login, but this file upload stuff still doesn't work. Basically what my problem seems to be (but I may be wrong) is that the data I send is not the same as that which would be sent has I actually uploaded a file, if that makes sense?

I checked out that article and we are basically doing the same thing, with the exception that he seems to use ASCII encoding for his final boundary and UTF8 for the preceeding ones, I can't see this making any difference, but if it could let me know and bag yourself some points (assuming that it fixes may problem)

Cheers
0
 
LVL 6

Expert Comment

by:der_jth
ID: 12333677
Another difference might be the fact that your content length is calculated differently. But please wait a moment; I've been toying around with the HTTP upload tonight. I'll post some code soon once I've tested it properly (during the next few hours).
0
 
LVL 6

Expert Comment

by:der_jth
ID: 12333880
Without analysing all your code (I believe the central issue is the mixture with strings, encodings et al), I delved into the issue and wrote some code myself. My blog entry at <http://www.heikniemi.net/hc/archives/000150.html> contains a better description and the original source in C#. I understand you want the answer in VB.net, so I quickly translated it. Please understand that I'm not a VB programmer; I did test the code below (it both works and compiles), but it's not likely to be a good example of VB. Also, it lacks all the comments. Please refer to the C# version for some more details.

I tested the code with some rudimentary PHP server side scripts. Please try this and see if it works for you. If it doesn't, it's likely that the other end of your transmission is doing some sort of hidden validation. If that's the case, the situation needs more thinking and working; perhaps it's a hidden form field they're expecting? Anyway, the code below should upload simple files and byte arrays nicely.

--
Imports System
Imports System.IO
Imports System.Net
Namespace JouniHeikniemi.Tools.Http

    Public Class HttpFileUpload

        Private Sub New()
        End Sub

        Public Structure UploadSpec
            Public Contents As Byte()
            Public FileName As String
            Public FieldName As String

            Public Sub New(ByVal contents As Byte(), ByVal fileName As String, ByVal fieldName As String)
                Me.Contents = contents
                Me.FileName = fileName
                Me.FieldName = fieldName
            End Sub

            Public Sub New(ByVal pathname As String, ByVal fieldName As String)
                Dim inFile As FileStream = New FileStream(pathname, FileMode.Open)
                Dim inBytes(inFile.Length) As Byte
                inFile.Read(inBytes, 0, inBytes.Length)
                CType(inFile, IDisposable).Dispose()
                Me.Contents = inBytes
                Me.FileName = Path.GetFileName(pathname)
                Me.FieldName = fieldName
            End Sub
        End Structure

        Public Shared Function UploadFile(ByVal pathname As String, ByVal url As String, ByVal fieldName As String, ByVal cookies As CookieContainer, ByVal credentials As CredentialCache) As HttpWebResponse
            Dim us(0) As UploadSpec
            us(0) = New UploadSpec(pathname, fieldName)
            Return Upload(url, cookies, credentials, us)
        End Function

        Public Shared Function UploadByteArray(ByVal data As Byte(), ByVal fileName As String, ByVal url As String, ByVal fieldName As String, ByVal cookies As CookieContainer, ByVal credentials As CredentialCache) As HttpWebResponse
            Dim us(0) As UploadSpec
            us(0) = New UploadSpec(data, fileName, fieldName)
            Return Upload(url, cookies, credentials, us)
        End Function

        Public Shared Function Upload(ByVal url As String, ByVal cookies As CookieContainer, ByVal credentials As CredentialCache, ByRef objects As UploadSpec()) As HttpWebResponse
            Dim req As HttpWebRequest = CType(WebRequest.Create(url), HttpWebRequest)
            If Not (cookies Is Nothing) Then
                req.CookieContainer = cookies
            End If
            If Not (credentials Is Nothing) Then
                req.Credentials = credentials
            End If
            Dim boundary As String = Guid.NewGuid.ToString.Replace("-", "")
            req.ContentType = "multipart/form-data; boundary=" + boundary
            req.Method = "POST"
            Dim postData As MemoryStream = New MemoryStream
            Dim newLine As String = "" & Microsoft.VisualBasic.Chr(13) & "" & Microsoft.VisualBasic.Chr(10) & ""
            Dim sw As StreamWriter = New StreamWriter(postData)
            For Each us As UploadSpec In objects
                sw.Write("--" + boundary + newLine)
                sw.Write("Content-Disposition: form-data; name=""{0}""; filename=""{1}""{2}", us.FieldName, us.FileName, newLine)
                sw.Write("Content-Type: application/octet-stream" + newLine + newLine)
                sw.Flush()
                postData.Write(us.Contents, 0, us.Contents.Length)
                sw.Write(newLine)
            Next
            sw.Write("--{0}--{1}", boundary, newLine)
            sw.Flush()
            req.ContentLength = postData.Length
            Dim s As Stream = req.GetRequestStream
            postData.WriteTo(s)
            CType(s, IDisposable).Dispose()
            postData.Close()
            Return CType(req.GetResponse, HttpWebResponse)
        End Function
    End Class
End Namespace
--
0
Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

 
LVL 6

Expert Comment

by:etmendz
ID: 12336843
You might need to coordinate with the owner of the URL and get their specification for the "file".

If their specification states that you should POST to their URL, then simply POST the file to them without any artificial formatting. Just POST the content as-is and use the properties of HttpWebRequest to set any HTTP header settings.

"Real file"? A receiver of a web POSTed file sees only a stream. It even has to fake receiving the filename using maybe a hidden form field for the filename or some property in the request. So maybe, you need to review their actual form and find the fields that the URL requires.

A successful POST should indicate a response code of 200. Otherwise, there's a problem.

Have fun.
0
 

Author Comment

by:duncanlatimer
ID: 12337431
der_jth, I have looked at your code and I am struggling to see the difference between what you are doing and what I do, I'm pretty sure this isn't an encoding issue, the length of my post buffer in bytes is exactly the same as the length of the string in chars and the buffer contains the chars in the string. I have tried changing the encoding to UTF8 and ASCII, neither worked.

etmendz, The response code I get is 200 and to the best of my knowledge the file format is specified correctly, however one problem with the site I am using is that uploading junk would also give me a 200.

This is the post data captured using HTTPLook from a genuine file upload:
POST /batch/edit_batch.php HTTP/1.1
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*
Referer: http://www.ibetx.com/batch/edit_batch.php
Accept-Language: en-gb
Content-Type: multipart/form-data; boundary=---------------------------7d41e252c04d2
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)
Host: www.ibetx.com
Content-Length: 389
Connection: Keep-Alive
Cache-Control: no-cache
Cookie: PHPSESSID=071a688b7aa19120dfaa732b8b6c2b0e

-----------------------------7d41e252c04d2
Content-Disposition: form-data; name="upfile"; filename="C:\Documents and Settings\Duncan\Desktop\sample.txt"
Content-Type: text/plain

~~,unmatched
U,95057,716682,35744013,L,2.00,1.00,Ali Deo
-----------------------------7d41e252c04d2
Content-Disposition: form-data; name="submit"

Upload
-----------------------------7d41e252c04d2--

And it is that post data that I am trying to emulate here, there is of course no actual file in my program, it is the content of a ficticious file if you like, my reference to a real file was simply trying to differenciate between what a browser would do when you uploaded a file that did exist locally and what my program did, does that clarify things (or make matters worse)?
0
 

Author Comment

by:duncanlatimer
ID: 12337448
I dunno if this helps explain the problem here but here is the actual code used to post currently:

Private Function httpPost(ByRef url As String, ByRef postData As String, ByRef cookie As String, ByRef contentType As String)

        Try
            Dim req As Net.HttpWebRequest = Net.WebRequest.Create(url)
            req.Method = "POST"
            req.Headers.Add("Cookie", cookie)
            req.ContentType = contentType
            Dim postBuff() As Byte = Text.Encoding.UTF8.GetBytes(postData)
            req.ContentLength = postBuff.Length
            Dim reqStream As IO.Stream = req.GetRequestStream()
            reqStream.Write(postBuff, 0, postBuff.Length)
            reqStream.Flush()
            reqStream.Close()
            Dim res As System.Net.HttpWebResponse = req.GetResponse()
            Dim rdr As New IO.StreamReader(res.GetResponseStream(), Text.Encoding.UTF8)
            httpPost = rdr.ReadToEnd()
            Console.WriteLine(res.StatusCode)
            res.Close()
        Catch e As Net.WebException
            Console.WriteLine("Woops, web exception thrown")
            httpPost = ""
        End Try

    End Function
0
 
LVL 6

Accepted Solution

by:
der_jth earned 500 total points
ID: 12337762
Now that you did post the actual file contents (text only), it certainly doesn't look like an encoding issue. Anyway, if posting it with my code doesn't work, then the receiving end is doing some validation you're failing. It can be something very trivial, ranging from PHPSESSID being wrong (outdated) to some form field issues as both I and etmendz have referred to earlier. Your posting code itself looks pretty fine, but the construction of postData is at least equally important. But on a quick look, it seems fine too.

Try to capture an actual post made with your browser and search for the differences?
0
 

Author Comment

by:duncanlatimer
ID: 12357473
Hi all, the answer was that the bondary in the data should be the boundary specified in the header preceded by two dashes. This was evident in both of der-jth's examples, but I didn't notice until I went through the POSTs byte by byte as suggested by der_jth, so he gets the points, perhaps a good tip for people wishing to do this kind of stuff would be don't use dashes in your boundary, it will then be more obvious what is going on, I did wonder why the code examples I looked at we adding the dashes to the begining of the boundary, and with hind sight................ but I know for the future
0

Featured Post

How to improve team productivity

Quip adds documents, spreadsheets, and tasklists to your Slack experience
- Elevate ideas to Quip docs
- Share Quip docs in Slack
- Get notified of changes to your docs
- Available on iOS/Android/Desktop/Web
- Online/Offline

Join & Write a Comment

This article explains how to create and use a custom WaterMark textbox class.  The custom WaterMark textbox class allows you to set the WaterMark Background Color and WaterMark text at design time.   IMAGE OF WATERMARKS STEPS Create VB …
It was really hard time for me to get the understanding of Delegates in C#. I went through many websites and articles but I found them very clumsy. After going through those sites, I noted down the points in a easy way so here I am sharing that unde…
Internet Business Fax to Email Made Easy - With eFax Corporate (http://www.enterprise.efax.com), you'll receive a dedicated online fax number, which is used the same way as a typical analog fax number. You'll receive secure faxes in your email, fr…
Excel styles will make formatting consistent and let you apply and change formatting faster. In this tutorial, you'll learn how to use Excel's built-in styles, how to modify styles, and how to create your own. You'll also learn how to use your custo…

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now