Simulating HTTP Post.

Hello,
I am developing a small utility, which will post a zip file to an application running on TOMCAT server.
On the server end, I have a listener which expects the zip file in a FormFile (a struts abstraction). The form field name for the is called "theFile".

On the client/utility end, I open a HttpsURLConnection, write the contents of the zip file into the stream and then close the connection. But, I do not know how to associate the name of the zip file with the contents I upload/write onto the stream. I am not sure, if header settings are correct as well.

I am pasting both the client and server code below. Thanks in advance.

/************** Client end ***************/

public void uploadCSV()
      {

                  try
                  {
                        productURLCSV = new URL("https://192.168.2.14:8443/productx/UploadCSV.do");
                  }
                  catch( MalformedURLException e )
                  {
                        System.out.println("MalformedURLException " + e );
                  }

                  connectionForUpload = null;
                  try
                  {
                        connectionForUpload = (HttpsURLConnection) productURLCSV.openConnection();
                  }
                  catch( IOException e )
                  {
                        System.out.println(" IOException in openConnection" + e );
                  }


                  /*
                        the Default verify method in JSSE seems to be failing. Provide an override, which will
                        always return true for the verify method.
                  */
                  MyHostnameVerifier DO_NOT_VERIFY1 = new MyHostnameVerifier();

                  connectionForUpload.setHostnameVerifier( DO_NOT_VERIFY1 );

                  connectionForUpload.setDoInput(true);

                  connectionForUpload.setDoOutput(true);

                  connectionForUpload.setAllowUserInteraction(false);

                  try
                  {
                        connectionForUpload.setRequestMethod("POST");
                  }
                  catch( ProtocolException e )
                  {
                        System.out.println(" ProtocolException ");
                  }

                  connectionForUpload.setUseCaches(false);

                  connectionForUpload.setRequestProperty("content-type", "application/zip");

                  //connectionForUpload.setRequestProperty("content-disposition", "form-data;name=theFile;filename=upload.zip");

                  PrintWriter sender = null;
                  OutputStream ops = null;

                  System.out.println( connectionForUpload );

                  try
                  {
                        ops = connectionForUpload.getOutputStream();
                  }
                  catch( IOException e )
                  {
                        System.out.println(" IOException getOutputStream " + e );
                  }

                  sender = new PrintWriter( ops );

                  /* open the zip file and write it to the stream */
                  File newFile = new File("upload.zip");
                  FileInputStream fin = null;
                  try
                  {
                        fin = new FileInputStream(newFile);
                  }
                  catch( FileNotFoundException e )
                  {
                  }

                  BufferedInputStream in = new BufferedInputStream(fin);
                  //Create a buffer for reading raw bytes from the file.
                  byte[] buf = new byte[1024];
                  //The len variable will keep track of how much we
                  //are able to actually read each time we try.
                  int len;
                  int totalSize = 0;

                  //Read from the file and write to the gzip archive.
                  try
                  {
                        while ( (len = in.read(buf)) >= 0 )
                        {
                               //diskFileStream.write(buf,0,len);
                               //sender.write(buf,0,len);
                               ops.write(buf,0,len);
                               totalSize += len;
                        }
                  }
                  catch( IOException e )
                  {
                        System.out.println(" IOException " + e );
                  }
                  System.out.println( totalSize + " bytes have been written to the stream " );


                  //sender.println("theFile=upload.zip");
                  //sender.close();

                  try
                  {
                        ops.flush();
                        ops.close();
                  }
                  catch( IOException e )
                  {
                        System.out.println(" IOException " + e );
                  }



                  BufferedReader reader;
                  InputStream ips = null;
                  try
                  {
                        ips = connectionForUpload.getInputStream();
                  }
                  catch( IOException e )
                  {
                        System.out.println(" IOException getInputStream " + e );
                  }

                  reader = new BufferedReader( new InputStreamReader( ips ) );

                  try
                  {
                        String returnMessage = reader.readLine();
                        System.out.println( "ReadLine returned :" + returnMessage );
                  }
                  catch( IOException e )
                  {
                        System.out.println(" IOException readline");
                  }

                  try
                  {
                        reader.close();
                  }
                  catch( IOException e )
                  {
                        System.out.println(" IOException close");
                  }

      }

/************** Client end ***************/

/************* Server end ***************/
private void processUploadFile(FormFile theFile, int propertyId, boolean toBeOverwritten)
            throws DataCorruptException, DataMissingException, DataInvalidException
      {
        String uploadFileName = theFile.getFileName();
            int size = theFile.getFileSize();

            String fileUploadDir
                  = StaticLookup.getPropertySystemParameterValue(propertyId, "PMS_INTG", "csvLocation");
            if(fileUploadDir.indexOf("\\") != -1)
                  fileUploadDir.replace('\\', File.separatorChar);
            if(fileUploadDir.indexOf("/") != -1)
                  fileUploadDir.replace('/', File.separatorChar);

      // throw DataMissingException if the upload directory for CSVs for given property is not available
            if(fileUploadDir.equals(""))
                  throw new DataMissingException("UploadAction:processUploadFile - Missing CSV upload location in DB for Property Id "+propertyId);

            File uploadedFile = new File(fileUploadDir+File.separator+uploadFileName);

            if(uploadedFile.exists() && (!toBeOverwritten))
                  throw new DataInvalidException("UploadAction:processUploadFile - Specified file already exists");

            try
            {
                  if(size > 0)
                  {
                        byte memoryfile[] = theFile.getFileData();
                        FileOutputStream diskFileStream = new FileOutputStream(uploadedFile);
                        diskFileStream.write(memoryfile);
                        diskFileStream.close();
                        if(size != uploadedFile.length())
                              throw new DataCorruptException("UploadAction:processUploadFile - Error writing file "+uploadFileName);
                  }
                  else
                        throw new DataCorruptException("UploadAction:processUploadFile - Specified file "+uploadFileName+" not found");

            }catch(FileNotFoundException fnfe){
                  throw new DataCorruptException("UploadAction:processUploadFile - Specified file "+uploadFileName+" not found");
            }catch(IOException ioexception){
                  throw new DataCorruptException("UploadAction:processUploadFile - Error reading file "+uploadFileName);
            }
      }

/************* Server end ***************/



shayadAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

heyhey_Commented:
in your case you will need to upload "multipart/form-data" POST request ... which is somewhat complicated.

much simpler solution will be to use "normal" POST (application/x-www-form-urlencoded). all you have to do is to send


"name=aa.txt&data=" + URLEncoder.encode(fileBytes);

0
heyhey_Commented:
for example on HTTPClient does it check

http://www.innovation.ch/java/HTTPClient/FAQ.html

http://www.innovation.ch/java/HTTPClient/api/HTTPClient/Codecs.html#mpFormDataEncode(HTTPClient.NVPair[], HTTPClient.NVPair[], HTTPClient.NVPair[])

(HTTPClient is URLConnection alternative, you can use it directly in your application)
0
Mick BarryJava DeveloperCommented:
> I do not know how to associate the name of the zip file
> with the contents I upload/write onto the stream.

just pass the name in the url and pass the data as you are already doing.
you could also pass the file details using the stream that you pass the data over, but in the url is probably easiest.
0
HTML5 and CSS3 Fundamentals

Build a website from the ground up by first learning the fundamentals of HTML5 and CSS3, the two popular programming languages used to present content online. HTML deals with fonts, colors, graphics, and hyperlinks, while CSS describes how HTML elements are to be displayed.

heyhey_Commented:
> just pass the name in the url and pass the data as you are already doing.

this won't work with FormFile at all

> the zip file in a FormFile (a struts abstraction)
0
shayadAuthor Commented:
Thanks for all the comments.

I have tried the approach suggested by heyhey, but it does not seem to work at all.

There are some observations though.
1. The response code is 200, which implies a success.

2. There is no data in the inputstream of the connection. I do not know what this means.

3. How can I figure out th response of the Server. My progress is slow because I am unable to figure out what is happening at the server end.

0
heyhey_Commented:
> I have tried the approach suggested by heyhey,


"normal" POST  ?

please post your code

> 2. There is no data in the inputstream of the connection. I do not know what this means.

not normal

> 3. How can I figure out th response of the Server. My progress is slow because I am unable to figure out what is happening at the server end.

you can use System.out.println() to print in server .log files or you can write to ServletRequest.getOutputStream() and print the data at the client

0
shayadAuthor Commented:
Hello again,
Thanks for you comments.
I have used normal POST. I have another observation:

If, I use a conventional browser to connect to my server, then the server prints out various messages in the log files such as "user shayad has been logged into the system".
However, if I use my own HttpClient, then no message is printed in the log files. TOMCAT's log files simply shows a message "session started [Timestamp]" and "session Terminated [TimeStamp]". I guess, these are printed in response to the URL.getConnection() API calls. But, I do not see any aplication specifc messages at all.

I guess, my login is failing to start with. I guess, the server should respond to a login request irrespective of the client it is talking to (the browser or my utility). Is it imperative for the server to return to response code. Thus, an empty return code is a sure shot sign that something is amiss.
But, the response code is 200, which implies a successful connection. I wonder why is the input stream empty.

SOS
Shayad.

PS:
I am working from home and do not have access to my code right away, but nothing has chnaged and all the core stuff is available in the question I posted y'day.
0
heyhey_Commented:
to sumamrize

1. you want to upload file from Java application to servlet / JSP page
2. my suggestion is to encode everything in a name/value pairs and upload it with HTTP POST.
3. you will have to create dedicated scipt that handles that POST (decodes the parameters, write data to file if needed).

try this technique on a simple servlet outside of your main code
0
Mick BarryJava DeveloperCommented:
Missed the struts part sorry :)

You need to rewrite your client to use "multipart/form-data" encoding.
0
Mick BarryJava DeveloperCommented:
here's some example code:

/**
* General solution to POST of multipart/form-data
* Essentially form fields with included attachments/uploads
*
* Add/Remove conetn dispositon and boundary sections as necessary
* Inlcudes exampels for form fileds , binary and text data file uploads
*
* * I have seen hundreds of postings regarding this geenral question
* of uploads, this client includes examples of the key points to
* being successfull at uploading multipart form-data.
* Add and or remove the construct below as necessary
* Bascially:
*
* Open Boundary
* contents dispositions
* Close Boundary
*
*
-----------------------------7d01ecf406a6
Content-Disposition: form-data; name="formField1"

value_of_field1
-----------------------------7d01ecf406a6
Content-Disposition: form-data; name="formField2"

value_of_field2
-----------------------------7d01ecf406a6
Content-Disposition: form-data; name="formField3"

value_of_field3
-----------------------------7d01ecf406a6
Content-Disposition: form-data; name="zipFileAttached"; filename="theZipFile"
Content-Type: application/x-zip-compressed

the binary data goes here
-----------------------------7d01ecf406a6
Content-Disposition: form-data; name="textFileAttached"; filename="theTextFile"
Content-Type: plain/text

the text data goes here
-----------------------------7d01ecf406a6--
*
* theUrlConnection.setRequestProperty( "Content-Type", "multipart/form-data; boundary=" + boundary );
* streamSend.append("Content-Disposition: form-data; name=\"formField\"\r\n");
* streamSend.append("\r\n");
* streamSend.append("value_of_field\r\n");
* streamSend.append("--" + boundary + "\r\n");
* streamSend.append("Content-Disposition: form-data; name=\"nextField\"\r\n");
* streamSend.append("\r\n");
* streamSend.append("value_of_next_field\r\n");
* streamSend.append("--" + boundary + "--");
*
*
*
* Helpful url: http://www.vivtek.com/rfc1867.html
* url: http://www.ietf.org/rfc/rfc1867.txt
*/
import java.net.*;
import java.io.*;
//import javax.net.*;
//import com.sun.net.ssl.*;
import java.util.*;
import java.text.*;
import java.security.*;

/**
* POST a multipart form-data http stream to servlet
*/
public class HttpMultiPartPost {
public static void main(String[] args) throws Exception {

/*

- Connect to Servlet and send

NAME="formField1" VALUE="value_of_field1"
NAME="formField2" VALUE="value_of_field2"
NAME="formField3" VALUE="value_of_field3"
NAME="zipFileAttached"
NAME="textFileAttached"

*/

URL destUrl = new URL("https://some.where.com/servlet/MailboxServlet");

// Prepare the connection for recieving the form
URLConnection theUrlConnection = destUrl.openConnection();
theUrlConnection.setDoOutput(true);
theUrlConnection.setDoInput(true);
theUrlConnection.setUseCaches(false);

// Multi-Part delimiter
String boundary="--7d021a37605f0";
theUrlConnection.setRequestProperty( "Content-Type", "multipart/form-data; boundary=" + boundary );

System.out.println("Before DataOutputStream");
DataOutputStream postFormFile = new DataOutputStream(theUrlConnection.getOutputStream());

// Buffer used to build stream of multi-part form field
StringBuffer streamSend = new StringBuffer();

// Each form field is sent within a boundary delimeter
// Each file is also sent within a boundary delimeter

// Start form fields boundary section
streamSend.append("--" + boundary + "\r\n");

// <input TYPE="HIDDEN" NAME="formField1" VALUE="value_of_field1">
streamSend.append("Content-Disposition: form-data; name=\"formField1\"\r\n");
streamSend.append("\r\n");
streamSend.append("value_of_field1\r\n");
streamSend.append("--" + boundary + "\r\n");

// <input TYPE="radio" NAME="formField2" VALUE="value_of_field2">
streamSend.append("Content-Disposition: form-data; name=\"formField2\"\r\n");
streamSend.append("\r\n");
streamSend.append("value_of_field2\r\n");
streamSend.append("--" + boundary + "\r\n");

// <select NAME="formField3" VALUE="value_of_field3">
streamSend.append("Content-Disposition: form-data; name=\"formField3\"\r\n");
streamSend.append("\r\n");
streamSend.append("value_of_field3\r\n");
streamSend.append("--" + boundary + "\r\n");

// End form fields boundary section

// Begin attachments/uploads or FILE portions of multipart form-data stream
// <input TYPE="FILE" NAME="zipFileAttached" VALUE="theZipfile">
streamSend.append("Content-Disposition: form-data;name=\"zipFileAttached\"; filename=\"theZipFile\"\r\n");
streamSend.append("Content-Type: application/x-zip-compressed\r\n");
streamSend.append("\r\n");
postFormFile.write(streamSend.toString().getBytes());
System.out.println("Data Stream\n"+streamSend.toString());

// Ok were ready for the binary "zip" data stream
System.out.println("Before zip data Send");
// Read/Write zip File as bytes
try {
FileInputStream uploadFileReader = new FileInputStream("c:\\path\\to\\the\\zipfile.zip");
int numBytesToRead=1024;
int availableBytesToRead;
availableBytesToRead=uploadFileReader.available();
System.out.println("size of file: " + availableBytesToRead);
while ((availableBytesToRead=uploadFileReader.available()) > 0) {
byte[] bufferBytesRead;
// Adjust size of buffered bytes if necessary
bufferBytesRead = availableBytesToRead >= numBytesToRead?
new byte[numBytesToRead]:
new byte[availableBytesToRead];
// Trap end of file condition
int numberOfBytesRead = uploadFileReader.read(bufferBytesRead);
// Did we reach end of file
if (numberOfBytesRead == -1) break;
// Write current buffered bytes to servlet
postFormFile.write(bufferBytesRead);
} // Iterate through contents of file
} catch (IOException e) {
e.printStackTrace(System.out);
}
streamSend = new StringBuffer();
streamSend.append("--" + boundary + "\r\n");
postFormFile.write(streamSend.toString().getBytes());
System.out.println("After zip data Send");
// Ok done sending zip file data stream


// plain text attachments or FILE portions of multipart form-data stream
// <input TYPE="FILE" NAME="textFileAttached" VALUE="theTextfile">
streamSend = new StringBuffer();
streamSend.append("Content-Disposition: form-data;name=\"textFileAttached\"; filename=\"theTextFile\"\r\n");
streamSend.append("Content-Type: text/plain\r\n");
streamSend.append("\r\n");
postFormFile.write(streamSend.toString().getBytes());
System.out.println("Data Stream\n"+streamSend.toString());

// Ok were ready for the binary "zip" data stream
System.out.println("Before text data Send");
// Read/Write zip File as bytes
try {
FileInputStream uploadFileReader = new FileInputStream("c:\\path\\to\\the\textFile.txt");
int numBytesToRead=1024;
int availableBytesToRead;
availableBytesToRead=uploadFileReader.available();
System.out.println("size of file: " + availableBytesToRead);
while ((availableBytesToRead=uploadFileReader.available()) > 0) {
byte[] bufferBytesRead;
// Adjust size of buffered bytes if necessary
bufferBytesRead = availableBytesToRead >= numBytesToRead?
new byte[numBytesToRead]:
new byte[availableBytesToRead];
// Trap end of file condition
int numberOfBytesRead = uploadFileReader.read(bufferBytesRead);
// Did we reach end of file
if (numberOfBytesRead == -1) break;
// Write current buffered bytes to servlet
postFormFile.write(bufferBytesRead);
} // Iterate through contents of file
} catch (IOException e) {
e.printStackTrace(System.out);
}
System.out.println("After text data Send");
// Ok done sending zip file data stream

// Terminate file multipart form-data POST boundary's
streamSend = new StringBuffer();
streamSend.append("--" + boundary + "--\r\n");
postFormFile.write(streamSend.toString().getBytes());
System.out.println("Data Stream\n"+streamSend.toString());
// close/cleanup the https output stream
postFormFile.flush();
postFormFile.close();

// Parse results to ensure file was sent ok."
// Expecting: "Some sort of html response/confirmation"
// Read response from File Upload "Send" post
// Initialize the input stream to be read from
InputStream responseFromDestUrl = theUrlConnection.getInputStream();
// Build up respnse into a buffer
StringBuffer thisResponsePage = new StringBuffer();
byte[] respBuffer = new byte[4096];
// The number of bytes read
int bytesRead=0;
// Iterate to build up response from the request using logonUrl
while ((bytesRead = responseFromDestUrl.read (respBuffer)) >= 0) {
thisResponsePage.append(new String(respBuffer).trim());
}
responseFromDestUrl.close(); // Close response stream
System.out.println(thisResponsePage.toString());

// Close the http response page stream
responseFromDestUrl.close();

System.out.println("After https stream close");

System.out.println ("HttpMultiPartPost Done.");
} // public static void main(String[] args) throws Exception
} // public class HttpMultiPartPost

 
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Mick BarryJava DeveloperCommented:
Thanks for the points :)
0
hspatelCommented:
objects@idg.
        I attempted you code, but I am having a problem uploading zip file created with WinZip, it seem one uploaded the resultant files are corrupt. They seem to be off by 2 bytes in size. Any suggestions. I compiled the code with Jdk 1.3.1

Thanks.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Java

From novice to tech pro — start learning today.