Solved

Problem unzipping file on Linux - need PK compat. 4.5

Posted on 2004-11-02
18,795 Views
Last Modified: 2012-08-13
Hi all,

I'm trying to unzip a VMware image on a Linux server. The zip file is 1.5G and when I try and unzip it, I get the following error message:

[root@server tmp]# unzip lnx111001.gz
Archive:  lnx111001.gz
   skipping: LNX111001-System-flat.vmdk  need PK compat. v4.5 (can do v2.1)

I've tried the latest version of unzip (5.51 from info-zip.org) but it gives the same error.

Can anyone advise how I can extract this file ?
0
Question by:Anonymouslemming
    20 Comments
     
    LVL 2

    Expert Comment

    by:Matt_Avery
    you probably want "gunzip" for a "*.gz" file, nout "unzip"
    0
     
    LVL 5

    Author Comment

    by:Anonymouslemming
    sorry - that was a mistype on my part - the file ends in .zip and was created with winzip
    0
     
    LVL 2

    Expert Comment

    by:ITG-SSNA
    Try running this on it, and post the output please:

    unzip -u
    unzip -t


    Thanks

    ~K Black
    0
     
    LVL 38

    Expert Comment

    by:wesly_chen
    Hi,

      Could you recompiling unzip with the "-DLARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64" option?
    This will allow the utilities to handle uncompressed data files greater than 2 GB in size, as long as the total size of the archive containing them is less than 2 GB. (The operating system also must have support for large files; for Linux, this involves the kernel [2.4 or later], the file system [at least Reiser and ext2 are supported], the C library [glibc 2.x], and possibly other file utilities [ls, rm, etc.] and the shell itself [bash, tcsh, etc.] if redirection is involved.)

       For more details, please check the following URL:
    http://www.info-zip.org/pub/infozip/FAQ.html

    Wesly
    0
     
    LVL 5

    Author Comment

    by:Anonymouslemming
    Unzip -u output:

    [pascow@server tmp]$ sudo unzip -u lnx111001.zip
    Archive:  lnx111001.zip
       skipping: LNX111001-System-flat.vmdk  need PK compat. v4.5 (can do v2.1)
     extracting: Windows Server 2003, Standard Edition.vmx.lck  
      inflating: LNX111001-System.vmdk  
      inflating: nvram                  
      inflating: vmware.log              
      inflating: Windows Server 2003, Standard Edition.vmx

    unzip -t output
    [pascow@server tmp]$ sudo unzip -t lnx111001.zip
    Archive:  lnx111001.zip
       skipping: LNX111001-System-flat.vmdk  need PK compat. v4.5 (can do v2.1)
        testing: Windows Server 2003, Standard Edition.vmx.lck   OK
        testing: LNX111001-System.vmdk    OK
        testing: nvram                    OK
        testing: vmware.log               OK
        testing: Windows Server 2003, Standard Edition.vmx   OK
    No errors detected in lnx111001.zip for the 5 files tested.
    1 file skipped because of unsupported compression or encoding.
    0
     
    LVL 5

    Author Comment

    by:Anonymouslemming
    Reply to Wesley Chan:

    I have tried compiling with those flags. The site mentioned did not make it clear how this should be done, so I edited the Makefile and changed line 776 to read
    CF="-O3 -Wall -I. -DASM_CRC $(LOC) -DLARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64"\

    I then compiled unzip, and saw the additional parameters passed to all of the GCC lines, for example:
    gcc -c -O3 -Wall -I. -DASM_CRC  -DLARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DSFX unzipsfx.c

    Now running this new unzip, firstly it just sits for quite a long time before doing anything. Here is the output of the -t run:

    [pascow@server tmp]$ ~pascow/tmp/unzip-5.51/unzip -t lnx111001.zip
    Archive:  lnx111001.zip
       skipping: LNX111001-System-flat.vmdk  need PK compat. v4.5 (can do v2.1)
        testing: Windows Server 2003, Standard Edition.vmx.lck   OK
        testing: LNX111001-System.vmdk       testing: nvram                  
      error:  invalid compressed data to inflate
        testing: vmware.log                  testing: Windows Server 2003, Standard Edition.vmx  At least one error was detected in lnx111001.zip.
    1 file skipped because of unsupported compression or encoding.

    This is different than the output I got before, but two things are notable...

    1. It still skips LNX111001-System-flat due to compat issues
    2. nvram now fails as being invalid compressed data

    -u just does nothing. No processer is being used, and nothing is updated to the screen.
    0
     
    LVL 38

    Expert Comment

    by:wesly_chen
    Hi,

       Could you try using gzip for Windows to compress it?
    You can download gzip here (version 1.3.5 for files > 4GB):
    http://gnuwin32.sourceforge.net/downlinks/gzip-bin.php

    You can check the following URL for more details
    http://gnuwin32.sourceforge.net/install.html

    Wesly
    0
     
    LVL 5

    Author Comment

    by:Anonymouslemming
    I can't control the compress side of the project unfortunately. These files are being supplied by a different business unit.
    0
     
    LVL 2

    Expert Comment

    by:Matt_Avery
    Any chance that the binary files in question have been trashed by FTPing them in "text" mode?

    You could rule this out by confirming that the size in bytes exactly matches on both systems, or (better) computing checksums.

    If you only have access to the unix version, you could check how many <CR> (0x0d) characters it contains. If the answer is zero, then it has probably been corrupted. (Would normally expect a compressed binary file to contain all 256 characters in roughly equal proportions. But text-mode FTP from Windows to unix will mistakenly remove <CR> characters).
    0
     
    LVL 5

    Author Comment

    by:Anonymouslemming
    The files themselves are fine. It's just a problem with zip files containing files over a certain size (uncompressed). This apparantly has to do with additional flags added by PKWare in PKZip 4.5 in 2001.

    I've done some testing, and even Winzip 8.x can't extract these files, but they extract fine on Winzip 9.0 - this appears to be new functionality in 9.

    Also, with regards to FTP and text mode, I've actually transfered the files to the machine using FTP, SCP and Samba when I thought I was getting corrupt files. The most recent files were SCP'd :)
    0
     
    LVL 38

    Expert Comment

    by:wesly_chen
    Or can you download to Windows machine and use winzip to uzip it. If it fails, then the zip file is corrupted.
    If it successes, then upload to Linux machine.

    Wesly
    0
     
    LVL 5

    Author Comment

    by:Anonymouslemming
    That would work, but there are 2 problems...

    1. The files are on our dev network. We don't have any windows boxes on that network, so the process would be burn DVD on dev, copy from DVD to windows box, buy external DVD writer for windows box, unzip files, burn to DVD, move back to dev.

    2. Even if we could transfer the files to a Windows machine over the network, this would mean moving 3GB over the network to the Windows box, then moving about 8.5GB back from the Windows box to the Linux machine. This is all over a 10Mb network as the machine in question isn't on the main switched network. This would be a little time consuming :)

    Ideally, I'd like to get this working on Linux. If that's not possible, it looks like my cheapest solution is a Maxtor external hard drive and doing this by USB. But that's money I'd rather not spend because of a limitation in unzip / a non-standard implementation in Winzip :)
    0
     
    LVL 2

    Expert Comment

    by:Matt_Avery
    OK, my apologies for distracting everyone with theories about the file being corrupted.

    The release notes at http://www.info-zip.org/UnZip.html say:

    "There may not ever be another major release of UnZip, although there might well be another minor release to add compatibility with PKWARE's large-file-support kludge"

    From the tone of this comment, it seems pretty clear that their version of "unzip" doesn't support the new PKWare extensions and isn't going to be upgraded in the near future.

    Anonymouslemming, I think you will have no choice but to push back on the other people in your organisation and demand the files in a different format. You could make noises about the "zip" format being "proprietary", "poorly documented" and "obsolescent".
    0
     
    LVL 38

    Expert Comment

    by:wesly_chen
    Hi,
       I agree with Matt_Avery. Push back to ask them to use gzip because of those troubles. ^_^

    Wesly
    0
     
    LVL 5

    Author Comment

    by:Anonymouslemming
    Matt_Avery:

    Thanks for the comments. Much as I'd love to push back with this, we're a windows shop, so any such noises would be met with hostility. As I'm responsible for trying to bring Linux onboard, I can't really ruffle too many feathers. I think this problem is going to be best resolved with a combination of a sneakernet and a mastercard :D

    Right - now for the important part - How do we close this down and split points ? I'm inclined to split them 50/50 between Matt_Avery and wesley_chen due to the ongoing attention that I've received from yourselves and the advice that you've given. I just don't really want this to become a featured solution.

    Would it be possible for you to both post something to the effect that this is not possible on Linux at this time due to the PKWare extensions and then I can accept those.
    0
     
    LVL 38

    Assisted Solution

    by:wesly_chen
    Well, according to http://www.info-zip.org/UnZip.html, it is not implemented yet to unzip big file (> 2GB) in the current version of unzip.
    However, it maybe some other commerical softwares can do it or can be done in the future.

    Wesly
    0
     
    LVL 2

    Expert Comment

    by:ITG-SSNA
    I suggest you check many of the file splitting utilities available on the web such as http://www.freebyte.com/hjsplit/ HJSplit

    You might be able to split larger files into mutliple parts, as a albeit dirty work-a-round for your problem.

    I think Winzip also suffers a 2GB limitation on locking a file to uncompress.

    Regards,

    ~K Black
    Irvine, Ca.
    0
     
    LVL 2

    Accepted Solution

    by:
    To summarise,

    we looked at the Info-Zip release notes: http://www.info-zip.org/UnZip.html
    and the FAQ http://www.info-zip.org/pub/infozip/FAQ.html
    (thanks to wesly_chen for pointing these out)

    It's clear that the version of "unzip" on Linux doesn't currently support archives containing files >2GB in size. Some Windows software (including WinZip 9.0) can deal with archives this size, but the mechanism for doing so is a poorly documented extension to the "zip" standard which is difficult for the Info-Zip developers to implement.

    As a Linux advocate, it annoys me to see an example where Windows appears to have better software available than Linux. However, since the Linux software is open source, at least it would be theoretically possible to fix it myself.

     
    0
     
    LVL 2

    Expert Comment

    by:ITG-SSNA
    I think before determining windows has better compression utilities, I'd consider the compression levels achieved by each of the utilities. It's commonly known that Unix compression utilities can reach a much higher level of compression with more options.

    Whether it suites you in this case shouldn't be made to reflect on the OS or the community, I think.

    My .02

    Regards

    ~KB
    0
     

    Expert Comment

    by:Relham_Vatsug
    p7zip would have worked for this.
    http://p7zip.sourceforge.net/
    0

    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    IT Security CISA, CISSP & CISM Certification

    Master the advanced techniques required to protect network resources from external threats with the IT Cyber Security bundle. Built around industry best-practice guidelines, the IT Cyber Security bundle consists of three in-depth courses.

    Little introduction about CP: CP is a command on linux that use to copy files and folder from one location to another location. Example usage of CP as follow: cp /myfoder /pathto/destination/folder/ cp abc.tar.gz /pathto/destination/folder/ab…
    The purpose of this article is to demonstrate how we can use conditional statements using Python.
    Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
    Connecting to an Amazon Linux EC2 Instance from Windows Using PuTTY.

    913 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    17 Experts available now in Live!

    Get 1:1 Help Now