Solved

Zip up file a file before sending it to the client.

Posted on 2004-08-28
8
568 Views
Last Modified: 2012-08-13
Hi all,

I'm using the Java Excel API from http://www.andykhan.com/jexcelapi/ and have a couple of questions. What I'm doing is creating an export to excel function that users can execute to retrieve large amounts of data through their browser with. It is currently working but I'm facing a couple of problems. First, this API only allows you to write the excel file in one go, which normally isn't a problem until you are exporting huge excel files. Some of my exports are ~20 MB's (and this is the way they want it so no arguing!) and with the default JVM setting it would crash with out of memory exceptions. I've increased the JVM's memory and tried to minimize memory usage elsewhere in the code to combat this problem.

What I'd like to do now is zip up the excel file before it's sent to the user but I'm having trouble figuring out how to compress the outputstream. Also, I would like to know how to set the size of the file in the response header but I can't figure out how to determine that with this API I'm using. I'm almost wondering if I should write the file to /temp on the server first, zip it up there, get the size info, and then send it to the user.

Here is the relevant part of the code. Any suggestions are greatly appreciated :)

response.setContentType("application/octet-stream");
response.setHeader( "Content-Disposition", "attachment; filename=Product-Export.xls" );

ServletOutputStream sos = response.getOutputStream();

WorkbookSettings settings = new WorkbookSettings();
settings.setEncoding("latin1");
WritableWorkbook workbook = Workbook.createWorkbook(sos, settings);
WritableSheet sheet = workbook.createSheet("Sheet 1", 0);

for (int i=0; i<fields.length; i++) {
    Label label = new Label(i, 0, (String)columnNames.get(fields[i]), headerFormat);
    sheet.addCell(label);
}

int lineNumber = 1;
ListIterator iterator = products.listIterator();
while (iterator.hasNext()) {
    Product thisProduct = (Product)iterator.next();
    for (int i=0; i<fields.length; i++) {
        createLabel(i, lineNumber, thisProduct, fields[i], sheet);
    }
    // Increment lineNumber each iteration and also get rid of the last product since we no longer need it.
    // Make a suggestion to the JVM every 1000 rounds that it needs to do some housework :P
    // (This has helped quite a bit with the memory problems).
    if (lineNumber++ % 1000 == 0) System.gc();
    iterator.remove();
}

workbook.write();
workbook.close();
0
Comment
Question by:pat5star
8 Comments
 
LVL 86

Expert Comment

by:CEHJ
Comment Utility
>>I would like to know how to set the size of the file in the response header but I can't figure out how to determine that with this API I'm using.


You wouldn't be able to if you were compressing it. Not setting the content length is not unusual though. Any online Javadoc for this API?
0
 
LVL 92

Expert Comment

by:objects
Comment Utility
No need to write it to a file, use something like:

OutputStream sos = new DeflaterOutputStream(response.getOutputStream());
0
 
LVL 18

Expert Comment

by:armoghan
Comment Utility
If it is possible using objects way then it would be best,
and you can make convert it to ZipOutputStream to send it in Zip format

http://javaalmanac.com/egs/java.util.zip/CreateZip.html
0
 
LVL 35

Expert Comment

by:girionis
Comment Utility
0
IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 
LVL 1

Accepted Solution

by:
mzhoreli earned 500 total points
Comment Utility
The suggestions are good. Except the Java Excel API you use doesn't work with java.util.zip.ZipOutputStream. It calls flush() on the stream and the servlet container sends response without data. And it doesn't close the stream.

As a workaround following class can be used:

import java.io.IOException;
import java.io.OutputStream;
import java.util.zip.ZipOutputStream;

public class NoFlushZipOutputStream extends ZipOutputStream {

      public NoFlushZipOutputStream(OutputStream out) {
            super(out);
      }
      
      public void flush() throws IOException {}
      
}


And here are the changes to the servlet code:

                response.setContentType("application/zip");
      response.setHeader( "Content-Disposition", "attachment; filename=Product-Export.zip" );
            
      NoFlushZipOutputStream sos = new NoFlushZipOutputStream(response.getOutputStream());
            
      ZipEntry ze = new ZipEntry("Product-Export.xls");
      sos.putNextEntry(ze);

                WorkbookSettings settings = new WorkbookSettings();
[...]
                workbook.write();
      workbook.close();
            
      sos.close();
0
 
LVL 1

Expert Comment

by:mzhoreli
Comment Utility
It looks like it is sufficient to properly close the java.util.ZipOutputStream at the end. No need to subclass it.
0
 
LVL 3

Author Comment

by:pat5star
Comment Utility
Thanks all for the suggestions :)

mzhoreli: Your solution worked perfectly! Thanks very much!

Since you seem familiar with this API I'm wondering what you thoughts are of it. I've used it for awhile now and love it because I find it easy to use and it's handled everything I've needed it to do up until now. I remember I first chose to use it because I found POI (I think that's what it was called) confusing. Now that I'm at a point where I need to write huge excel files quite often I wonder if I need to consider using another package. Any thoughts?

Thanks again,

-Pat
0
 
LVL 3

Author Comment

by:pat5star
Comment Utility
Your right, I changed it and it still works the same. Thanks again,

-Pat
0

Featured Post

Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

Join & Write a Comment

For customizing the look of your lightweight component and making it look lucid like it was made of glass. Or: how to make your component more Apple-ish ;) This tip assumes your component to be of rectangular shape and completely opaque. (COD…
Java had always been an easily readable and understandable language.  Some relatively recent changes in the language seem to be changing this pretty fast, and anyone that had not seen any Java code for the last 5 years will possibly have issues unde…
Viewers will learn about basic arrays, how to declare them, and how to use them. Introduction and definition: Declare an array and cover the syntax of declaring them: Initialize every index in the created array: Example/Features of a basic arr…
This tutorial covers a step-by-step guide to install VisualVM launcher in eclipse.

763 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now