Solved

Zip up file a file before sending it to the client.

Posted on 2004-08-28
8
593 Views
Last Modified: 2012-08-13
Hi all,

I'm using the Java Excel API from http://www.andykhan.com/jexcelapi/ and have a couple of questions. What I'm doing is creating an export to excel function that users can execute to retrieve large amounts of data through their browser with. It is currently working but I'm facing a couple of problems. First, this API only allows you to write the excel file in one go, which normally isn't a problem until you are exporting huge excel files. Some of my exports are ~20 MB's (and this is the way they want it so no arguing!) and with the default JVM setting it would crash with out of memory exceptions. I've increased the JVM's memory and tried to minimize memory usage elsewhere in the code to combat this problem.

What I'd like to do now is zip up the excel file before it's sent to the user but I'm having trouble figuring out how to compress the outputstream. Also, I would like to know how to set the size of the file in the response header but I can't figure out how to determine that with this API I'm using. I'm almost wondering if I should write the file to /temp on the server first, zip it up there, get the size info, and then send it to the user.

Here is the relevant part of the code. Any suggestions are greatly appreciated :)

response.setContentType("application/octet-stream");
response.setHeader( "Content-Disposition", "attachment; filename=Product-Export.xls" );

ServletOutputStream sos = response.getOutputStream();

WorkbookSettings settings = new WorkbookSettings();
settings.setEncoding("latin1");
WritableWorkbook workbook = Workbook.createWorkbook(sos, settings);
WritableSheet sheet = workbook.createSheet("Sheet 1", 0);

for (int i=0; i<fields.length; i++) {
    Label label = new Label(i, 0, (String)columnNames.get(fields[i]), headerFormat);
    sheet.addCell(label);
}

int lineNumber = 1;
ListIterator iterator = products.listIterator();
while (iterator.hasNext()) {
    Product thisProduct = (Product)iterator.next();
    for (int i=0; i<fields.length; i++) {
        createLabel(i, lineNumber, thisProduct, fields[i], sheet);
    }
    // Increment lineNumber each iteration and also get rid of the last product since we no longer need it.
    // Make a suggestion to the JVM every 1000 rounds that it needs to do some housework :P
    // (This has helped quite a bit with the memory problems).
    if (lineNumber++ % 1000 == 0) System.gc();
    iterator.remove();
}

workbook.write();
workbook.close();
0
Comment
Question by:pat5star
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
8 Comments
 
LVL 86

Expert Comment

by:CEHJ
ID: 11920397
>>I would like to know how to set the size of the file in the response header but I can't figure out how to determine that with this API I'm using.


You wouldn't be able to if you were compressing it. Not setting the content length is not unusual though. Any online Javadoc for this API?
0
 
LVL 92

Expert Comment

by:objects
ID: 11920594
No need to write it to a file, use something like:

OutputStream sos = new DeflaterOutputStream(response.getOutputStream());
0
 
LVL 18

Expert Comment

by:armoghan
ID: 11920714
If it is possible using objects way then it would be best,
and you can make convert it to ZipOutputStream to send it in Zip format

http://javaalmanac.com/egs/java.util.zip/CreateZip.html
0
Get 15 Days FREE Full-Featured Trial

Benefit from a mission critical IT monitoring with Monitis Premium or get it FREE for your entry level monitoring needs.
-Over 200,000 users
-More than 300,000 websites monitored
-Used in 197 countries
-Recommended by 98% of users

 
LVL 1

Accepted Solution

by:
mzhoreli earned 500 total points
ID: 11921361
The suggestions are good. Except the Java Excel API you use doesn't work with java.util.zip.ZipOutputStream. It calls flush() on the stream and the servlet container sends response without data. And it doesn't close the stream.

As a workaround following class can be used:

import java.io.IOException;
import java.io.OutputStream;
import java.util.zip.ZipOutputStream;

public class NoFlushZipOutputStream extends ZipOutputStream {

      public NoFlushZipOutputStream(OutputStream out) {
            super(out);
      }
      
      public void flush() throws IOException {}
      
}


And here are the changes to the servlet code:

                response.setContentType("application/zip");
      response.setHeader( "Content-Disposition", "attachment; filename=Product-Export.zip" );
            
      NoFlushZipOutputStream sos = new NoFlushZipOutputStream(response.getOutputStream());
            
      ZipEntry ze = new ZipEntry("Product-Export.xls");
      sos.putNextEntry(ze);

                WorkbookSettings settings = new WorkbookSettings();
[...]
                workbook.write();
      workbook.close();
            
      sos.close();
0
 
LVL 1

Expert Comment

by:mzhoreli
ID: 11922049
It looks like it is sufficient to properly close the java.util.ZipOutputStream at the end. No need to subclass it.
0
 
LVL 3

Author Comment

by:pat5star
ID: 11922056
Thanks all for the suggestions :)

mzhoreli: Your solution worked perfectly! Thanks very much!

Since you seem familiar with this API I'm wondering what you thoughts are of it. I've used it for awhile now and love it because I find it easy to use and it's handled everything I've needed it to do up until now. I remember I first chose to use it because I found POI (I think that's what it was called) confusing. Now that I'm at a point where I need to write huge excel files quite often I wonder if I need to consider using another package. Any thoughts?

Thanks again,

-Pat
0
 
LVL 3

Author Comment

by:pat5star
ID: 11922104
Your right, I changed it and it still works the same. Thanks again,

-Pat
0

Featured Post

Online Training Solution

Drastically shorten your training time with WalkMe's advanced online training solution that Guides your trainees to action. Forget about retraining and skyrocket knowledge retention rates.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

INTRODUCTION Working with files is a moderately common task in Java.  For most projects hard coding the file names, using parameters in configuration files, or using command-line arguments is sufficient.   However, when your application has vi…
Introduction This article is the first of three articles that explain why and how the Experts Exchange QA Team does test automation for our web site. This article explains our test automation goals. Then rationale is given for the tools we use to a…
Viewers will learn about the different types of variables in Java and how to declare them. Decide the type of variable desired: Put the keyword corresponding to the type of variable in front of the variable name: Use the equal sign to assign a v…
This video teaches viewers about errors in exception handling.
Suggested Courses

623 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question