Solved

Zip up file a file before sending it to the client.

Posted on 2004-08-28
8
577 Views
Last Modified: 2012-08-13
Hi all,

I'm using the Java Excel API from http://www.andykhan.com/jexcelapi/ and have a couple of questions. What I'm doing is creating an export to excel function that users can execute to retrieve large amounts of data through their browser with. It is currently working but I'm facing a couple of problems. First, this API only allows you to write the excel file in one go, which normally isn't a problem until you are exporting huge excel files. Some of my exports are ~20 MB's (and this is the way they want it so no arguing!) and with the default JVM setting it would crash with out of memory exceptions. I've increased the JVM's memory and tried to minimize memory usage elsewhere in the code to combat this problem.

What I'd like to do now is zip up the excel file before it's sent to the user but I'm having trouble figuring out how to compress the outputstream. Also, I would like to know how to set the size of the file in the response header but I can't figure out how to determine that with this API I'm using. I'm almost wondering if I should write the file to /temp on the server first, zip it up there, get the size info, and then send it to the user.

Here is the relevant part of the code. Any suggestions are greatly appreciated :)

response.setContentType("application/octet-stream");
response.setHeader( "Content-Disposition", "attachment; filename=Product-Export.xls" );

ServletOutputStream sos = response.getOutputStream();

WorkbookSettings settings = new WorkbookSettings();
settings.setEncoding("latin1");
WritableWorkbook workbook = Workbook.createWorkbook(sos, settings);
WritableSheet sheet = workbook.createSheet("Sheet 1", 0);

for (int i=0; i<fields.length; i++) {
    Label label = new Label(i, 0, (String)columnNames.get(fields[i]), headerFormat);
    sheet.addCell(label);
}

int lineNumber = 1;
ListIterator iterator = products.listIterator();
while (iterator.hasNext()) {
    Product thisProduct = (Product)iterator.next();
    for (int i=0; i<fields.length; i++) {
        createLabel(i, lineNumber, thisProduct, fields[i], sheet);
    }
    // Increment lineNumber each iteration and also get rid of the last product since we no longer need it.
    // Make a suggestion to the JVM every 1000 rounds that it needs to do some housework :P
    // (This has helped quite a bit with the memory problems).
    if (lineNumber++ % 1000 == 0) System.gc();
    iterator.remove();
}

workbook.write();
workbook.close();
0
Comment
Question by:pat5star
8 Comments
 
LVL 86

Expert Comment

by:CEHJ
ID: 11920397
>>I would like to know how to set the size of the file in the response header but I can't figure out how to determine that with this API I'm using.


You wouldn't be able to if you were compressing it. Not setting the content length is not unusual though. Any online Javadoc for this API?
0
 
LVL 92

Expert Comment

by:objects
ID: 11920594
No need to write it to a file, use something like:

OutputStream sos = new DeflaterOutputStream(response.getOutputStream());
0
 
LVL 18

Expert Comment

by:armoghan
ID: 11920714
If it is possible using objects way then it would be best,
and you can make convert it to ZipOutputStream to send it in Zip format

http://javaalmanac.com/egs/java.util.zip/CreateZip.html
0
Announcing the Most Valuable Experts of 2016

MVEs are more concerned with the satisfaction of those they help than with the considerable points they can earn. They are the types of people you feel privileged to call colleagues. Join us in honoring this amazing group of Experts.

 
LVL 35

Expert Comment

by:girionis
ID: 11920770
0
 
LVL 1

Accepted Solution

by:
mzhoreli earned 500 total points
ID: 11921361
The suggestions are good. Except the Java Excel API you use doesn't work with java.util.zip.ZipOutputStream. It calls flush() on the stream and the servlet container sends response without data. And it doesn't close the stream.

As a workaround following class can be used:

import java.io.IOException;
import java.io.OutputStream;
import java.util.zip.ZipOutputStream;

public class NoFlushZipOutputStream extends ZipOutputStream {

      public NoFlushZipOutputStream(OutputStream out) {
            super(out);
      }
      
      public void flush() throws IOException {}
      
}


And here are the changes to the servlet code:

                response.setContentType("application/zip");
      response.setHeader( "Content-Disposition", "attachment; filename=Product-Export.zip" );
            
      NoFlushZipOutputStream sos = new NoFlushZipOutputStream(response.getOutputStream());
            
      ZipEntry ze = new ZipEntry("Product-Export.xls");
      sos.putNextEntry(ze);

                WorkbookSettings settings = new WorkbookSettings();
[...]
                workbook.write();
      workbook.close();
            
      sos.close();
0
 
LVL 1

Expert Comment

by:mzhoreli
ID: 11922049
It looks like it is sufficient to properly close the java.util.ZipOutputStream at the end. No need to subclass it.
0
 
LVL 3

Author Comment

by:pat5star
ID: 11922056
Thanks all for the suggestions :)

mzhoreli: Your solution worked perfectly! Thanks very much!

Since you seem familiar with this API I'm wondering what you thoughts are of it. I've used it for awhile now and love it because I find it easy to use and it's handled everything I've needed it to do up until now. I remember I first chose to use it because I found POI (I think that's what it was called) confusing. Now that I'm at a point where I need to write huge excel files quite often I wonder if I need to consider using another package. Any thoughts?

Thanks again,

-Pat
0
 
LVL 3

Author Comment

by:pat5star
ID: 11922104
Your right, I changed it and it still works the same. Thanks again,

-Pat
0

Featured Post

Courses: Start Training Online With Pros, Today

Brush up on the basics or master the advanced techniques required to earn essential industry certifications, with Courses. Enroll in a course and start learning today. Training topics range from Android App Dev to the Xen Virtualization Platform.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

By the end of 1980s, object oriented programming using languages like C++, Simula69 and ObjectPascal gained momentum. It looked like programmers finally found the perfect language. C++ successfully combined the object oriented principles of Simula w…
Java Flight Recorder and Java Mission Control together create a complete tool chain to continuously collect low level and detailed runtime information enabling after-the-fact incident analysis. Java Flight Recorder is a profiling and event collectio…
Viewers will learn about arithmetic and Boolean expressions in Java and the logical operators used to create Boolean expressions. We will cover the symbols used for arithmetic expressions and define each logical operator and how to use them in Boole…
This tutorial explains how to use the VisualVM tool for the Java platform application. This video goes into detail on the Threads, Sampler, and Profiler tabs.

816 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now