Unexpected end of ZLIB input stream error

Hi:

My program is throwing "java.io.EOFException: Unexpected end of ZLIB input stream" error every now and then. The tricky part of this is that it's not reproduceable. I schedule the program to run every night. When I test the program next morning the exception is thrown it usually runs fine. Please help.

My code:

URL zipurl = new URL(myurl);
ZipInputStream zin = new ZipInputStream(zipurl.openStream());
ZipEntry ze;
   
while ((ze = zin.getNextEntry())!=null) {                                  //exception is thrown at here
      if (ze.getName().startsWith(someString)){
          InputStream in = new BufferedInputStream(zin);
          Reader isr = new InputStreamReader(in);
          BufferedReader br = new BufferedReader(isr);
   
          while ((line =br.readLine())!=null && line.trim().length()!= 0) {

           ...

           }

       }

}
sdc248Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

CEHJCommented:
Am i getting deja vu (with you sdc248)?
0
sdc248Author Commented:
yes, I thought the: line.trim().length()!=0
in the nested while loop had resolved the problem. It didn't. :<
0
CEHJCommented:
I've a feeling this could be tricky. Can you please post the stack trace?
0
Cloud Class® Course: Python 3 Fundamentals

This course will teach participants about installing and configuring Python, syntax, importing, statements, types, strings, booleans, files, lists, tuples, comprehensions, functions, and classes.

x4uCommented:
Maybe the zipfile itself was truncated for some reason.

>> URL zipurl = new URL(myurl);

What kind of url is this? Are you sure that it can be read entirely in all cases?
0
objectsCommented:
Don't use a BufferedReader, and instead read from the input stream directecty ensuring you don't read more bytes than are in the ZipEntry.
And write the bytes to a ByteArrayOutputStream to get the bytes, you can then use a BufferedReader on the array of bytes read to read your uncompressed text data.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
objectsCommented:
so two steps:
1. Read zip entry into byte array
2. Process the byte array as requried

Let me know if you have any questions.
0
x4uCommented:
>> ensuring you don't read more bytes than are in the ZipEntry.

ZipInputStream takes care of this already
0
objectsCommented:
Not in earlier versions, what version are you running it against?
Another thing to try would be to upgrade your JRE.
0
CEHJCommented:
>>Don't use a BufferedReader

That's certainly good advice if not all entries are text files - is that the case?

>>and instead read from the input stream directecty ensuring you don't read more bytes than are in the ZipEntry.

How would reading from the stream directly make any difference?
0
CEHJCommented:
Try reading exactly entry size bytes from the entry
0
objectsCommented:
> Try reading exactly entry size bytes from the entry

please don't repeat previous posts, it just confuses things.
0
objectsCommented:
> That's certainly good advice if not all entries are text files - is that the case?

and the type of file is irrelevent
0
CEHJCommented:
>>please don't repeat previous posts, it just confuses things.

LOL. Which previous post was that exactly? If i had done, it wouldn't 'confuse' things - it would reinforce them
0
CEHJCommented:
while ((ze = zin.getNextEntry())!=null && bytesRead <= ze.getSize())            
0
objectsCommented:
> ensuring you don't read more bytes than are in the ZipEntry.

helps to read previous posts, in fact you are expected to.
0
CEHJCommented:
>>helps to read previous posts, in fact you are expected to.

I've not only read it, but commented on that piece of 'advice'. I notice you haven't answered my question about it ...
0
WebstormCommented:
Hi sdc248,

You need to call the closeEntry() method:

while ((ze = zin.getNextEntry())!=null) {                                  //exception is thrown at here
      if (ze.getName().startsWith(someString)){
          InputStream in = new BufferedInputStream(zin);
          Reader isr = new InputStreamReader(in);
          BufferedReader br = new BufferedReader(isr);
   
          while ((line =br.readLine())!=null && line.trim().length()!= 0) {

           ...

           }
           zin.closeEntry(); // ready to read next entry
       }

}
0
sdc248Author Commented:
Hi guys, thanks for the input. Regarding your questions:

1. stack trace: I didn't take it (damn!)
2. the url and the zip file both look ok, although I don't know what kind of zip program the file publisher uses. I read from somewhere that Jave (old version, maybe) has bug while dealing with zip file generated by certain zip program and that it tries to read beyond the input stream.
3. my jdk/jre version is 1.5.0
4. the zip file in question contains about 30 files in it. They are csv files. I am not sure how a csv file differs from a text file, except that there are commas in between tokens.
5. my program has to process the content of the csv file, line by line. For example, for every line, first token is the date, 2nd token is the location, 3rd to 27 are the prices from hour 1 to hour 24 for that day. Sometimes part of the prices could be missing. By using BufferedInputStream I could read one line, parse it, process token by token, if anything worng, throw exception, if not next line. It's easy and safe to me. objects advice might work, but just a little unclear to me. For one thing, how many bytes should I read from the byte array each time? Plus, now that the exception is thrown at the code that is getting zip entries (it hasn't started to read the file yet), I wonder whether that is the solution?



0
CEHJCommented:
>>They are csv files. I am not sure how a csv file differs from a text file

They are text files

Did you try Webstorm's suggestion?
0
sdc248Author Commented:
i certainly have implmented Webstorm's suggestion. But I am not sure whether this is the solution since the exception was thrown even before the if block starts...
0
CEHJCommented:
On the first pass through the loop?
0
WebstormCommented:
>> But I am not sure whether this is the solution since the exception was thrown even before the if block starts...

Then you can also try to put   zin.closeEntry();   outside the if, just before the end of the while loop.

But as you get it the file from network, you may get wrong zip file. Try to save it to a file before reading this file to test if it's a real zip file. In case you got wrong data from network, you may catch the exception and retry to download it again.
0
objectsCommented:
> For one thing, how many bytes should I read from the byte array each time

as many as u want as long as its less than the amout remaining in the entry.
Just keep track of the amount read from each entry, that way you know how many bytes are remaining.
0
sdc248Author Commented:
probably not the first pass as I remember there was a pause (of those outputs of my System.out.println()s)  before the error was thrown.

Copying the zip file is probably a good idea. I could use it to re-produce the problem for later testing.

Will put another zin.closeEntry() as suggested by Webstorm. Guess have to wait for a few weeks to see whether it really works.
0
objectsCommented:
> Copying the zip file is probably a good idea. I could use it to re-produce the problem for later testing.

And you could delete the copy on success
0
CEHJCommented:
>>For one thing, how many bytes should I read from the byte array each time?

See http:Q_21792744.html#16319271
0
sdc248Author Commented:
Hi:

Sorry for belating reply but I got tied up with the other projects. Since I don't know when I could come back to this issue I am closing the thread for now and spliting the points equally to you guys. Thank you very much for the efforts to help. I will open a new thread if I have further questions.
0
CEHJCommented:
:-)
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Java

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.