?
Solved

2 Gig File Size Limit????

Posted on 2002-06-23
10
Medium Priority
?
1,330 Views
Last Modified: 2012-06-21
I seem to be running to a 2 gig file size limit on a new linux machine.  I'm trying to unzip a file that I created on a Windows machine (under NTFS).  Once upzipped, the file should be over 3 gigabytes.  However, unzip always fails at 2.1478 gigs.....and I've tried other methods.

Is there some inherent file size restriction? If so, how do I get rid of it?

I have a Dell-installed Redhat Linux 7.3 server.  

Thanks,
Brett
0
Comment
Question by:brgordon
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
10 Comments
 
LVL 1

Expert Comment

by:elniniokev
ID: 7102510
AFAIK there is a 2 gig file size limit for Linux on 32-bit machines.  There should be a patch available to increase the max file size to 4 gig from Redhat.  You will have to recompile your kernel to take advantage of the patch.
0
 

Author Comment

by:brgordon
ID: 7102589
Is there any way to get past the 4 gig limit?

Thanks,
Brett
0
 
LVL 17

Expert Comment

by:dorward
ID: 7103225
Patch your kernel to support the XFS file system, install the XFS tools, and create an XFS file system on one of your partitions, depending on the page size the maximum file size varies beteen sixteen and sixty four Terrabytes, which should be more then enough for most people.

http://oss.sgi.com/projects/xfs/

Bonus - its journeling :)
0
How to Create Failover DNS Record Sets in Route 53

Route 53 has the ability to easily configure DNS record sets specifically for failover scenarios. These failover record sets can be configured to failover to full-blown deployments in other regions or to a static HTML page that informs your customers of the issue.

 
LVL 51

Expert Comment

by:ahoffmann
ID: 7104087
With kernel 2.4.x and glibc 2.2.x there is no longer a 2GB limit.
Unfortunatelly most distribution come with the old binaries compiled with a (g)libc with the 2GB limit, and so won't work proper on 2.4.x kernels.
You need programs compiled under 2.4.x, which also use the open64(), lseek() and so on ... , function.
AFAIK there is a LFS (large filesystem) fileutils package.
0
 
LVL 51

Expert Comment

by:ahoffmann
ID: 7104101
BTW, if you compile tar:

    ./configure --enable-largefile

0
 

Author Comment

by:brgordon
ID: 7104130
ahoffman & dorward,

Thanks for both of your responses.  As it turns out, the problem does not seem to be with the OS - it seems like 'unzip' has a problem with > 2gb files.  I was able to create a 10 gb file.  

Do certain compression utilities have problems with the 2 gb limit?  If so, how do I get around it?  Are there other utilities?

ahoffman - could the 'unzip' utility be one of the old binaries?

Thanks,
Brett
0
 

Author Comment

by:brgordon
ID: 7104137
ahoffman,

lastly, what does 'AFAIK' mean?

-brett
0
 
LVL 51

Accepted Solution

by:
ahoffmann earned 400 total points
ID: 7104947
AFAIK - as far as I know
(OOTFA - one of these f... acronyms :)

> could the 'unzip' utility be one of the old binaries?
yes.
You can check with:
  nm -gop `which unzip`|grep -i open
but unzip is most likely a stripped binary, so you probably get nothing

I suggest to use gzip instead, but mae shure that it is 2GB aware:
  nm -gop `gzip`|grep -i open
(if there is a reference to glib2.0, it is a old one)
0
 

Author Comment

by:brgordon
ID: 7105064
I was able to use 'zcat'.

Thanks,
Brett
0
 
LVL 51

Expert Comment

by:ahoffmann
ID: 7105109
zcat is a link to gzip, usually ;-)
0

Featured Post

Get MySQL database support online, now!

At Percona’s web store you can order your MySQL database support needs in minutes. No hassles, no fuss, just pick and click. Pay online with a credit card.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

SSH (Secure Shell) - Tips and Tricks As you all know SSH(Secure Shell) is a network protocol, which we use to access/transfer files securely between two networked devices. SSH was actually designed as a replacement for insecure protocols that sen…
It’s 2016. Password authentication should be dead — or at least close to dying. But, unfortunately, it has not traversed Quagga stage yet. Using password authentication is like laundering hotel guest linens with a washboard — it’s Passé.
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.
Suggested Courses
Course of the Month10 days, 14 hours left to enroll

770 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question