Solved

2 Gig File Size Limit????

Posted on 2002-06-23
10
1,316 Views
Last Modified: 2012-06-21
I seem to be running to a 2 gig file size limit on a new linux machine.  I'm trying to unzip a file that I created on a Windows machine (under NTFS).  Once upzipped, the file should be over 3 gigabytes.  However, unzip always fails at 2.1478 gigs.....and I've tried other methods.

Is there some inherent file size restriction? If so, how do I get rid of it?

I have a Dell-installed Redhat Linux 7.3 server.  

Thanks,
Brett
0
Comment
Question by:brgordon
10 Comments
 
LVL 1

Expert Comment

by:elniniokev
Comment Utility
AFAIK there is a 2 gig file size limit for Linux on 32-bit machines.  There should be a patch available to increase the max file size to 4 gig from Redhat.  You will have to recompile your kernel to take advantage of the patch.
0
 

Author Comment

by:brgordon
Comment Utility
Is there any way to get past the 4 gig limit?

Thanks,
Brett
0
 
LVL 17

Expert Comment

by:dorward
Comment Utility
Patch your kernel to support the XFS file system, install the XFS tools, and create an XFS file system on one of your partitions, depending on the page size the maximum file size varies beteen sixteen and sixty four Terrabytes, which should be more then enough for most people.

http://oss.sgi.com/projects/xfs/

Bonus - its journeling :)
0
 
LVL 51

Expert Comment

by:ahoffmann
Comment Utility
With kernel 2.4.x and glibc 2.2.x there is no longer a 2GB limit.
Unfortunatelly most distribution come with the old binaries compiled with a (g)libc with the 2GB limit, and so won't work proper on 2.4.x kernels.
You need programs compiled under 2.4.x, which also use the open64(), lseek() and so on ... , function.
AFAIK there is a LFS (large filesystem) fileutils package.
0
 
LVL 51

Expert Comment

by:ahoffmann
Comment Utility
BTW, if you compile tar:

    ./configure --enable-largefile

0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 

Author Comment

by:brgordon
Comment Utility
ahoffman & dorward,

Thanks for both of your responses.  As it turns out, the problem does not seem to be with the OS - it seems like 'unzip' has a problem with > 2gb files.  I was able to create a 10 gb file.  

Do certain compression utilities have problems with the 2 gb limit?  If so, how do I get around it?  Are there other utilities?

ahoffman - could the 'unzip' utility be one of the old binaries?

Thanks,
Brett
0
 

Author Comment

by:brgordon
Comment Utility
ahoffman,

lastly, what does 'AFAIK' mean?

-brett
0
 
LVL 51

Accepted Solution

by:
ahoffmann earned 100 total points
Comment Utility
AFAIK - as far as I know
(OOTFA - one of these f... acronyms :)

> could the 'unzip' utility be one of the old binaries?
yes.
You can check with:
  nm -gop `which unzip`|grep -i open
but unzip is most likely a stripped binary, so you probably get nothing

I suggest to use gzip instead, but mae shure that it is 2GB aware:
  nm -gop `gzip`|grep -i open
(if there is a reference to glib2.0, it is a old one)
0
 

Author Comment

by:brgordon
Comment Utility
I was able to use 'zcat'.

Thanks,
Brett
0
 
LVL 51

Expert Comment

by:ahoffmann
Comment Utility
zcat is a link to gzip, usually ;-)
0

Featured Post

Complete Microsoft Windows PC® & Mac Backup

Backup and recovery solutions to protect all your PCs & Mac– on-premises or in remote locations. Acronis backs up entire PC or Mac with patented reliable disk imaging technology and you will be able to restore workstations to a new, dissimilar hardware in minutes.

Join & Write a Comment

It’s 2016. Password authentication should be dead — or at least close to dying. But, unfortunately, it has not traversed Quagga stage yet. Using password authentication is like laundering hotel guest linens with a washboard — it’s Passé.
Join Greg Farro and Ethan Banks from Packet Pushers (http://packetpushers.net/podcast/podcasts/pq-show-93-smart-network-monitoring-paessler-sponsored/) and Greg Ross from Paessler (https://www.paessler.com/prtg) for a discussion about smart network …
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now