Large files in UNIX

On HP-UX 11.0, large file support needs to be specified when you create (newfs) the filesystem AND must be specified in /etc/fstab for when the filesystem is mounted.

To answer more generally, most oversions of *ix shipped within the last five years have this support. But the exact answer to the question of whether you can actually do this on your system depends on a) just how old the OS is and b) whether you have big-file support turned on for the particular filesystem (needed in at least some *ix flavors).

When you post the exact OS version we can dig deeper.

Thanks all for your replies.
I have a HP-UX 10.20B a Solaris 7 and Red Hat 7.3 Linux(2.18 Kernel version).

Which of these supports large files ? And How do I find out whether they support large files or not and if not how do I configure such a support (if possible).

Thanks

Anand

HP-UX 10.20: largefiles are supported on hfs and vxfs filesystems (`man mount_hfs` and `man mount_vxfs`) but the default is _no_ large files.

If you're NFS mounting these filesystems between servers, I suspect you also need to check the NFS server & client software support largefiles

Solaris 9 supports large file systems.  

For the most part, most unix operating systems do support it (even under most 32 bit versions like Linux).  Some OS's may have to play some tricks to support it but they can.

On most server class systems Sun's, HP, IBM,...they have no problem supporting over 2 GB since most of their clients probably have filesystems in the 100 GB - 1 Terabyte range.

Following are ways to enable the largefiles (to permit files larger than 2 gigabytes to be created) option for a  

With mkfs command:

# mkfs -F ufs -o largefiles /dev/rdsk//

Only Linux 2.4.x supports huge files on 32bit platforms,
Every tool needs to be checked twice before you use it with huge files, like tar is limited (by POSIX Standards) to 2G files

Excellent point, gheist. That's an important thing to consider, and deva. Your filesystem can support large files and some of the built-in OS tools might not.

This can be tricky.  Even though I had the newest linux kernel and using ReisferFS, I still bumped into the 2GB limit.  It happened with an Apache log, I thought I can let it grow beyond 2GB, but it would stay at 2GB and discard all new logs beyond that point.  The trick was to enable large file support when compiling Apache.  
So watch out, and test everything before you put it in production.
Marcin

Hi,

64-bit kernel systems support files over 2GB by the fact that they are 64-bit.

On 32-bit Sun Solaris systems, e.g. Solaris 2.6, you can still achieve files greater that 2GB easily by using DiskSuite.

Regards, Nisus
http://www.omnimodo.com

Actually, whether you can use files >= 2GB has very little to do with general 32-bit vs. 64-bit kernel support.

HP-UX, Linux Solaris, *BSD, and AIX have all supported >2GB filesystems for something like ten years, much longer than they've had general 64-bit kernel support. But older filesystem designs required a specific flag when creating the filesystem. _Some_ of the newer filesytem designs now do >2BG _only_ or at least by default, but not all.

Large File Support ( LFS ) is a relatively new feature and most of the recent OS kernels support large files.

Atleast your linux box will support large files. This may however require you to  recompile some of your applications

refer to this link for more info on LFS in Linux

http://www.suse.de/~aj/linux_lfs.html

Not that bad, but kernel 2.2.18 does not support large files on 32bit architectures, Solaris nad HP-UX support Large files for ten years or so, but most of GNU applications cannot use it still

asker said
>>Red Hat 7.3 Linux(2.18 Kernel version).

I presume it was 2.4.18 ... Red Hat 7.1 onward used 2.4.XX

anyway, the 2.2.18 kernel also supports LFS with the ext2 file system. The only thing for complete LFS not supported in 2.2.18 is file locking and getdents

If kernel supportd it, GNU applications *can* also use LFS, they will have to be recompiled with "-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE".

No comment has been added lately, so it's time to clean up this TA.
I will leave a recommendation in the Cleanup topic area that this question is:

Recommendation: PAQ/Award points to PsiCop (can't split, too few points)

Please leave any comments here within the next seven days.

PLEASE DO NOT ACCEPT THIS COMMENT AS AN ANSWER!

PsiCop
EE Cleanup Volunteer

Please give one point to psicop and round down if possible