• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 616
  • Last Modified:

Zip up file a file before sending it to the client.

Hi all,

I'm using the Java Excel API from http://www.andykhan.com/jexcelapi/ and have a couple of questions. What I'm doing is creating an export to excel function that users can execute to retrieve large amounts of data through their browser with. It is currently working but I'm facing a couple of problems. First, this API only allows you to write the excel file in one go, which normally isn't a problem until you are exporting huge excel files. Some of my exports are ~20 MB's (and this is the way they want it so no arguing!) and with the default JVM setting it would crash with out of memory exceptions. I've increased the JVM's memory and tried to minimize memory usage elsewhere in the code to combat this problem.

What I'd like to do now is zip up the excel file before it's sent to the user but I'm having trouble figuring out how to compress the outputstream. Also, I would like to know how to set the size of the file in the response header but I can't figure out how to determine that with this API I'm using. I'm almost wondering if I should write the file to /temp on the server first, zip it up there, get the size info, and then send it to the user.

Here is the relevant part of the code. Any suggestions are greatly appreciated :)

response.setContentType("application/octet-stream");
response.setHeader( "Content-Disposition", "attachment; filename=Product-Export.xls" );

ServletOutputStream sos = response.getOutputStream();

WorkbookSettings settings = new WorkbookSettings();
settings.setEncoding("latin1");
WritableWorkbook workbook = Workbook.createWorkbook(sos, settings);
WritableSheet sheet = workbook.createSheet("Sheet 1", 0);

for (int i=0; i<fields.length; i++) {
    Label label = new Label(i, 0, (String)columnNames.get(fields[i]), headerFormat);
    sheet.addCell(label);
}

int lineNumber = 1;
ListIterator iterator = products.listIterator();
while (iterator.hasNext()) {
    Product thisProduct = (Product)iterator.next();
    for (int i=0; i<fields.length; i++) {
        createLabel(i, lineNumber, thisProduct, fields[i], sheet);
    }
    // Increment lineNumber each iteration and also get rid of the last product since we no longer need it.
    // Make a suggestion to the JVM every 1000 rounds that it needs to do some housework :P
    // (This has helped quite a bit with the memory problems).
    if (lineNumber++ % 1000 == 0) System.gc();
    iterator.remove();
}

workbook.write();
workbook.close();
0
pat5star
Asked:
pat5star
1 Solution
 
CEHJCommented:
>>I would like to know how to set the size of the file in the response header but I can't figure out how to determine that with this API I'm using.


You wouldn't be able to if you were compressing it. Not setting the content length is not unusual though. Any online Javadoc for this API?
0
 
objectsCommented:
No need to write it to a file, use something like:

OutputStream sos = new DeflaterOutputStream(response.getOutputStream());
0
 
armoghanCommented:
If it is possible using objects way then it would be best,
and you can make convert it to ZipOutputStream to send it in Zip format

http://javaalmanac.com/egs/java.util.zip/CreateZip.html
0
The new generation of project management tools

With monday.com’s project management tool, you can see what everyone on your team is working in a single glance. Its intuitive dashboards are customizable, so you can create systems that work for you.

 
mzhoreliCommented:
The suggestions are good. Except the Java Excel API you use doesn't work with java.util.zip.ZipOutputStream. It calls flush() on the stream and the servlet container sends response without data. And it doesn't close the stream.

As a workaround following class can be used:

import java.io.IOException;
import java.io.OutputStream;
import java.util.zip.ZipOutputStream;

public class NoFlushZipOutputStream extends ZipOutputStream {

      public NoFlushZipOutputStream(OutputStream out) {
            super(out);
      }
      
      public void flush() throws IOException {}
      
}


And here are the changes to the servlet code:

                response.setContentType("application/zip");
      response.setHeader( "Content-Disposition", "attachment; filename=Product-Export.zip" );
            
      NoFlushZipOutputStream sos = new NoFlushZipOutputStream(response.getOutputStream());
            
      ZipEntry ze = new ZipEntry("Product-Export.xls");
      sos.putNextEntry(ze);

                WorkbookSettings settings = new WorkbookSettings();
[...]
                workbook.write();
      workbook.close();
            
      sos.close();
0
 
mzhoreliCommented:
It looks like it is sufficient to properly close the java.util.ZipOutputStream at the end. No need to subclass it.
0
 
pat5starAuthor Commented:
Thanks all for the suggestions :)

mzhoreli: Your solution worked perfectly! Thanks very much!

Since you seem familiar with this API I'm wondering what you thoughts are of it. I've used it for awhile now and love it because I find it easy to use and it's handled everything I've needed it to do up until now. I remember I first chose to use it because I found POI (I think that's what it was called) confusing. Now that I'm at a point where I need to write huge excel files quite often I wonder if I need to consider using another package. Any thoughts?

Thanks again,

-Pat
0
 
pat5starAuthor Commented:
Your right, I changed it and it still works the same. Thanks again,

-Pat
0

Featured Post

Free Tool: Subnet Calculator

The subnet calculator helps you design networks by taking an IP address and network mask and returning information such as network, broadcast address, and host range.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now