ZIP file format issue

Hello

We are experiencing some issues with third party generated  ZIP files.

Although WinZip or 7zip actually manage to unzip them (while complaining about format errors) Linux built-in utilities will silently (!) fail with them.

Although I have already raised the issue with the provider it would be very useful if someone with proper expertise could let me know exactly what's wrong with the files we receive (see attached exemple).
5a2fe9b0d9f22.zip
LVL 2
Alexandre TakacsCTOAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Bill PrewIT / Software Engineering ConsultantCommented:
Well, I've never delved too deep into the bytes of a ZIP file, and there are several flavors of ZIP as well, but this one clearly has an issue.  It was either created bad, or suffered a change in transmission.  zipinfo reports the below.  7z can sort of open it, but also reports it as a malformed file.  And the Content.xml file seems to be unreadable.

Archive:  5a2fe9b0d9f22.zip
Zip file size: 36174 bytes, number of entries: 18
warning [5a2fe9b0d9f22.zip]:  1 extra byte at beginning or within zipfile
  (attempting to process anyway)
-rw----     2.0 fat     1850 b- defN 17-Dec-12 15:37 dd6c27469b9f45a781bee78d5b78569e.Res00002.xml
-rw----     2.0 fat      174 b- defN 17-Dec-12 15:37 957e729710a448608be2f5cad89d8934.ResultStatistics.xml
-rw----     2.0 fat     5436 b- defN 17-Dec-12 15:37 019ab81539934887860c7a4176afb016.2017-12-12_Z53_CH190024024041964900T_CHF_475274.xml
-rw----     2.0 fat     2307 b- defN 17-Dec-12 15:37 129438d9bcb54277ac2130fdb0d951fb.2017-12-12_Z53_CH670024024041964901Z_CHF_475268.xml
-rw----     2.0 fat     7417 b- defN 17-Dec-12 15:37 aeac1ee267c74b8f934b3e9e26ca1ab9.2017-12-12_Z53_CH190024024041964900T_CHF_475278.xml
-rw----     2.0 fat     3618 b- defN 17-Dec-12 15:37 e613efe4f434444ba49414cadb7b843c.2017-12-12_Z53_CH190024024041964900T_CHF_475277.xml
-rw----     2.0 fat     8386 b- defN 17-Dec-12 15:37 a6c764d81df042ecb6d75042a0e46d3c.2017-12-12_Z53_CH190024024041964900T_CHF_475276.xml
-rw----     2.0 fat     6084 b- defN 17-Dec-12 15:37 be315ff7c91d49a5b032a849ab3c3b65.2017-12-12_Z53_CH190024024041964900T_CHF_475271.xml
-rw----     2.0 fat    14530 b- defN 17-Dec-12 15:37 ae7f2bd929df4100ae488858f674ea3d.Res00001.xml
-rw----     2.0 fat     2119 b- defN 17-Dec-12 15:37 41a5445820f34127936a30405434c63d.Req00001.xml
-rw----     2.0 fat     9725 b- defN 17-Dec-12 15:37 5bc42064a5c84222b0d2645203f4a96e.2017-12-12_Z53_CH190024024041964900T_CHF_475269.xml
-rw----     2.0 fat     2611 b- defN 17-Dec-12 15:37 0205806a3c32491b99dbf7b8c48a3448.2017-12-12_Z53_CH670024024041964901Z_CHF_475270.xml
-rw----     2.0 fat     4479 b- defN 17-Dec-12 15:37 c93a010601574058b07b8d5aa552b062.2017-12-12_Z53_CH190024024041964900T_CHF_475272.xml
-rw----     2.0 fat     1538 b- defN 17-Dec-12 15:37 2f25c36d2fad4aef82be3f847ed5151f.Req00002.xml
-rw----     2.0 fat     3327 b- defN 17-Dec-12 15:37 15e9c0fad4f04d589f0c4b3b0d505aff.2017-12-12_Z53_CH670024024041964901Z_CHF_475273.xml
-rw----     2.0 fat    12334 b- defN 17-Dec-12 15:37 41ed4916e4c3466fa12c59f7bf7c3cfe.2017-12-12_Z53_CH190024024041964900T_CHF_475267.xml
-rw----     2.0 fat     2386 b- defN 17-Dec-12 15:37 57edbf39fc1c49f5a2a18249c3d91436.2017-12-12_Z53_CH190024024041964900T_CHF_475275.xml
-rw----     2.0 fat     8419 b- defN 17-Dec-12 15:37 Content.xml
18 files, 96740 bytes uncompressed, 32279 bytes compressed:  66.6%

Open in new window


Running a repair with WINZIP reports errors as expected, but not a lot of detail.

WinZip(R) Command Line Support Add-On Version 4.0 64-bit (Build 10562)
Copyright (c) 1991-2013 WinZip International LLC - All Rights Reserved


fixing   Zip file 5a2fe9b0d9f22.zip
    scanning dd6c27469b9f45a781bee78d5b78569e.Res00002.xml
    scanning 957e729710a448608be2f5cad89d8934.ResultStatistics.xml
    scanning 019ab81539934887860c7a4176afb016.2017-12-12_Z53_CH190024024041964900T_CHF_475274.xml
    scanning 129438d9bcb54277ac2130fdb0d951fb.2017-12-12_Z53_CH670024024041964901Z_CHF_475268.xml
    scanning aeac1ee267c74b8f934b3e9e26ca1ab9.2017-12-12_Z53_CH190024024041964900T_CHF_475278.xml
    scanning e613efe4f434444ba49414cadb7b843c.2017-12-12_Z53_CH190024024041964900T_CHF_475277.xml
    scanning a6c764d81df042ecb6d75042a0e46d3c.2017-12-12_Z53_CH190024024041964900T_CHF_475276.xml
    scanning be315ff7c91d49a5b032a849ab3c3b65.2017-12-12_Z53_CH190024024041964900T_CHF_475271.xml
    scanning ae7f2bd929df4100ae488858f674ea3d.Res00001.xml
    scanning 41a5445820f34127936a30405434c63d.Req00001.xml
    scanning 5bc42064a5c84222b0d2645203f4a96e.2017-12-12_Z53_CH190024024041964900T_CHF_475269.xml
    scanning 0205806a3c32491b99dbf7b8c48a3448.2017-12-12_Z53_CH670024024041964901Z_CHF_475270.xml
    scanning c93a010601574058b07b8d5aa552b062.2017-12-12_Z53_CH190024024041964900T_CHF_475272.xml
    scanning 2f25c36d2fad4aef82be3f847ed5151f.Req00002.xml
    scanning 15e9c0fad4f04d589f0c4b3b0d505aff.2017-12-12_Z53_CH670024024041964901Z_CHF_475273.xml
    scanning 41ed4916e4c3466fa12c59f7bf7c3cfe.2017-12-12_Z53_CH190024024041964900T_CHF_475267.xml
    scanning 57edbf39fc1c49f5a2a18249c3d91436.2017-12-12_Z53_CH190024024041964900T_CHF_475275.xml
    scanning Content.xml
Warning: some information from central directory headers may have been lost
    copying: dd6c27469b9f45a781bee78d5b78569e.Res00002.xml
    copying: 957e729710a448608be2f5cad89d8934.ResultStatistics.xml
    copying: 019ab81539934887860c7a4176afb016.2017-12-12_Z53_CH190024024041964900T_CHF_475274.xml
    copying: 129438d9bcb54277ac2130fdb0d951fb.2017-12-12_Z53_CH670024024041964901Z_CHF_475268.xml
    copying: aeac1ee267c74b8f934b3e9e26ca1ab9.2017-12-12_Z53_CH190024024041964900T_CHF_475278.xml
    copying: e613efe4f434444ba49414cadb7b843c.2017-12-12_Z53_CH190024024041964900T_CHF_475277.xml
    copying: a6c764d81df042ecb6d75042a0e46d3c.2017-12-12_Z53_CH190024024041964900T_CHF_475276.xml
    copying: be315ff7c91d49a5b032a849ab3c3b65.2017-12-12_Z53_CH190024024041964900T_CHF_475271.xml
    copying: ae7f2bd929df4100ae488858f674ea3d.Res00001.xml
    copying: 41a5445820f34127936a30405434c63d.Req00001.xml
    copying: 5bc42064a5c84222b0d2645203f4a96e.2017-12-12_Z53_CH190024024041964900T_CHF_475269.xml
    copying: 0205806a3c32491b99dbf7b8c48a3448.2017-12-12_Z53_CH670024024041964901Z_CHF_475270.xml
    copying: c93a010601574058b07b8d5aa552b062.2017-12-12_Z53_CH190024024041964900T_CHF_475272.xml
    copying: 2f25c36d2fad4aef82be3f847ed5151f.Req00002.xml
    copying: 15e9c0fad4f04d589f0c4b3b0d505aff.2017-12-12_Z53_CH670024024041964901Z_CHF_475273.xml
    copying: 41ed4916e4c3466fa12c59f7bf7c3cfe.2017-12-12_Z53_CH190024024041964900T_CHF_475267.xml
    copying: 57edbf39fc1c49f5a2a18249c3d91436.2017-12-12_Z53_CH190024024041964900T_CHF_475275.xml
    copying: Content.xml
creating Zip file 5a2fe9b0d9f22FIXED.zip


Zip file: 5a2fe9b0d9f22FIXED.zip

Testing dd6c27469b9f45a781bee78d5b78569e.Res00002.xml   OK
Testing 957e729710a448608be2f5cad89d8934.ResultStatistics.xml   OK
Testing 019ab81539934887860c7a4176afb016.2017-12-12_Z53_CH190024024041964900T_CHF_475274.xml   OK
Testing 129438d9bcb54277ac2130fdb0d951fb.2017-12-12_Z53_CH670024024041964901Z_CHF_475268.xml   OK
Testing aeac1ee267c74b8f934b3e9e26ca1ab9.2017-12-12_Z53_CH190024024041964900T_CHF_475278.xml   OK
Testing e613efe4f434444ba49414cadb7b843c.2017-12-12_Z53_CH190024024041964900T_CHF_475277.xml   OK
Testing a6c764d81df042ecb6d75042a0e46d3c.2017-12-12_Z53_CH190024024041964900T_CHF_475276.xml   OK
Testing be315ff7c91d49a5b032a849ab3c3b65.2017-12-12_Z53_CH190024024041964900T_CHF_475271.xml   OK
Testing ae7f2bd929df4100ae488858f674ea3d.Res00001.xml   OK
Testing 41a5445820f34127936a30405434c63d.Req00001.xml   OK
Testing 5bc42064a5c84222b0d2645203f4a96e.2017-12-12_Z53_CH190024024041964900T_CHF_475269.xml   OK
Testing 0205806a3c32491b99dbf7b8c48a3448.2017-12-12_Z53_CH670024024041964901Z_CHF_475270.xml   OK
Testing c93a010601574058b07b8d5aa552b062.2017-12-12_Z53_CH190024024041964900T_CHF_475272.xml   OK
Testing 2f25c36d2fad4aef82be3f847ed5151f.Req00002.xml   OK
Testing 15e9c0fad4f04d589f0c4b3b0d505aff.2017-12-12_Z53_CH670024024041964901Z_CHF_475273.xml   OK
Testing 41ed4916e4c3466fa12c59f7bf7c3cfe.2017-12-12_Z53_CH190024024041964900T_CHF_475267.xml   OK
Testing 57edbf39fc1c49f5a2a18249c3d91436.2017-12-12_Z53_CH190024024041964900T_CHF_475275.xml   OK
Testing Content.xml             Error: Invalid compressed data to expand (inflate) the file
At least one error was detected in 5a2fe9b0d9f22FIXED.zip.

Open in new window


Clearly the file has a problem.  Find out how they are creating it and get them to correct, not a lot you can really do after the fact...


»bp
Alexandre TakacsCTOAuthor Commented:
Thanks

That's pretty much how far I got. If anyone with more knowledge of ZIP format would chime in it will be appreciated!
David FavorLinux/LXD/WordPress/Hosting SavantCommented:
This all depends on version of Linux you're using + version of unzip installed.

If you're using something recent, like Ubuntu Zesty or Artful, then the zip installed will utilize all new technologies, like support for large zipfiles + decryption + decompression of alternative zip compressor formats.

If you're using an old Linux, which will only have an old version of unzip available for install, then likely you'll have a good bit of trouble decompressing modern .zip files.
Determine the Perfect Price for Your IT Services

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden with our free interactive tool and use it to determine the right price for your IT services. Download your free eBook now!

Bill PrewIT / Software Engineering ConsultantCommented:
While that's true for un*x, since the ZIP files are reported as corrupted using current versions of WinZip and 7-Zip on Windows, I would still be looking at the files themselves, I think they are the root of the problem.


»bp
David FavorLinux/LXD/WordPress/Hosting SavantCommented:
You can also try the following command. The more recent the version of zip, the better this will work...

zip -FF broken.zip --out fixed.zip

Open in new window

Alexandre TakacsCTOAuthor Commented:
Thanks for all the advices

I would still be very interested by some analysis of the file provided (ie what is actually wrong or questionable with it's structure)
Bill PrewIT / Software Engineering ConsultantCommented:
I'm taking a further look at this, I think I see why there is a problem, but I haven't finished probing the ZIP data to try and see exactly where it is coming in.  Interestingly, the file doesn't seem to be in bad shape, looks like maybe just a 1 byte discrepancy, but need more time to dig in.  Right now my paying consulting is needing my attention, but I want to look more at this, if only for my personal knowledge...


»bp
Alexandre TakacsCTOAuthor Commented:
Thanks!
Scott SilvaNetwork AdministratorCommented:
What I see so far is an unconfirmed start of data, which is a header issue. Also there is data after the end of the payload.
Is there a chance these are being moved by FTP in ascii mode? Or their zip program or temp file system is bad...
Bill PrewIT / Software Engineering ConsultantCommented:
Okay, after a bit more time with the ZIP file in a HEX ediitor, here's what I see as the problems.

First problem I see:

The End of central directory record signature starts at 0x8D38.  It reports the offset to the central directory to be at 0x8517, however the first signiture of a file record in the central directory occurs at offset 0x8518.

Second problem I observe, and likely related to above:

Both the central directory file entry, and the local file header report the compressed data length of the Content.xml file as 0x5C8, but in looking at the compressed data associated with this file (based on the location of the next signature in the ZIP) the actual compressed data seems to be ox5C9 long.

It's a little interesting that the last entry in the file is the one involved in the problem, I wonder if that file was added to the ZIP as a second step, and perhaps their utility is not adding a new file quite right.  Really hard to know the root cause of the problem without the original files, etc.


»bp

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
serialbandCommented:
Linux zip does not support newer winzip formats.  Just install the 7zip command line and run that.
Bill PrewIT / Software Engineering ConsultantCommented:
The ZIP isn't a "new" flavor of zip, looks to be a simple basic zip format.  No zip utility is going to be able to extract the Content.xml file, there seems to be corruption related to that.

Alexandre Takacs, have you been able to talk to the supplier of the zip?  Do they see the corruption on there end, or is it only after transmission to you?


»bp
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Linux

From novice to tech pro — start learning today.