lseek64 - 8 gigabyte limit?

How can I lseek64 to 8 gigabytes or beyond?

I am opening my file (well, raw device) with open64, and with the O_LARGEFILE flag.  My lseek64's are being called with SEEK_SET.  I am using full-fledged off64_t's, not off_t's.

lseek64 calls seem to be alright up to an 8 gigabyte barrier, at which point, the position I try to seek to seems to be getting modulused by 8 gigabytes.  i.e. 8 gigs seeks to 0, 8 gigs + 1k seeks to 1k, and so on.  At least, that is what the return values of lseek64 tell me.

I have googled all over, and google-groups'ed for solutions, but any slightly relevant discussion is about the 2- or 4-gig limit of regular lseek.

This is on a Red Hat 7 hybrid, running a modified 2.4.18 kernel (with SMP), compiled with "cc (GCC) 3.2.2 20030222".  My program runs on a reiserfs filesystem.
PistosChristouAsked:
Who is Participating?
 
Computer101Commented:
PAQed, with points refunded (200)

Computer101
E-E Admin
0
 
shivsaCommented:
u can check this link and see if u get any hint.
http://www.suse.de/~aj/linux_lfs.html
0
 
PistosChristouAuthor Commented:
shivsa: I've half-read, half-skimmed it, but it does not help my problem.
0
Cloud Class® Course: Microsoft Exchange Server

The MCTS: Microsoft Exchange Server 2010 certification validates your skills in supporting the maintenance and administration of the Exchange servers in an enterprise environment. Learn everything you need to know with this course.

 
sunnycoderCommented:
I never heard/encountred this milit for lseek64 ... may be because I never had to seek so far ...
If you are sure this is the problem, then a solution can be

while ( seek_length > 8 gb )
{
        lseek64 ( 8gb );
        seek_length = seek_length - 8gb;
}

lseek64 ( seek_length );  
0
 
PistosChristouAuthor Commented:
sunnycoder: I will try incremental seeking as you suggest.  For the moment, I am doing:

off64_t position_desired = (some number > 8 gigs);
off64_t actual_position = lseek64( fd, position_desired, SEEK_SET );

and then (actual_position != position_position).
0
 
PistosChristouAuthor Commented:
Okay, incremental seeking doesn't help.  If I increment in 512 MB increments, it seeks fine at 512, 1024 and 1536 MB, but hitting 2 gigs causes it to return -2 gigs as a return value, leading me to believe that off64_t is being typed as a 32-bit signed integer, as opposed to the long long that it is supposed to be.
0
 
shivsaCommented:
Dear  PistosChristou,

there are lots of projects goin in LARGEFILE support areas, and this area is not yet stable.
till now linux has good stable support for files upto 4GB and but not above than that.

please check the site for new developements and all.
http://www.scyld.com/products/beowulf/software/lfs-5.html
http://www.suse.de/~aj/linux_lfs.html


good luck !
0
 
sunnycoderCommented:
In that case, I am afraid you have not either used 64 interface consistently or you have not compiled the program with appropriate flags

you need to Compile your programs with "gcc -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE".
also make sure that you use the O_LARGEFILE flag with open to operate on large files
0
 
PistosChristouAuthor Commented:
sunnycoder: I have done a search through my source code, and every relevant open is an open64, and every lseek is an lseek64.  Each open has the O_LARGEFILE flag, and my compiler flags include: "-D_LARGEFILE64_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64".

shivsa: Thank you for the links.

In case it is of relevance, here is my glibc version:

$ /lib/libc.so.6
GNU C Library stable release version 2.3.2, by Roland McGrath et al.
Copyright (C) 2003 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 3.2.2 20030222 (Red Hat Linux 3.2.2-5).
Compiled on a Linux 2.4.20 system on 2003-02-27.
0
 
PistosChristouAuthor Commented:
Further possibly relevant information:

The program is being compiled on a different machine than the one it's being executed on.
0
 
sunnycoderCommented:
>The program is being compiled on a different machine than the one it's being executed on.
try compiling and running on the same machine and post the results

> causes it to return -2 gigs as a return value,
is suspect ... I am not so sure about lseek64 but lseek returns -1 on failure and a positive value otherwise ... I believe lseek64 must not be too much different ....
0
 
PistosChristouAuthor Commented:
Compiling on the same machine doesn't bear any fruit.

> > causes it to return -2 gigs as a return value,
> is suspect ... I am not so sure about lseek64 but lseek returns -1 on failure and a positive value otherwise ... I believe lseek64 must not be too much different ....

Yes, I know, so this makes things rather, as you say, suspect.  Perhaps some high-order or low-order bits are being shifted, truncated, zeroed... who knows.  I've tried casting every relevant operand of every relevant operation, but it doesn't help.  The number being passed in is validly > 2 gigs, but the number being spit out is not.

Is it a possibility that my 64-bit calls and data types are being transformed at compile time to 32-bit versions, due to some header or library being missing or corrupt?
0
 
PistosChristouAuthor Commented:
Okay, someone else working on the project has come to the conclusion that the problem is a bug in glibc, wherein lseek64, though performing the seek correctly, does not return more than 32 bits.  A thread that we found:

http://mail.gnu.org/archive/html/bug-glibc/2003-10/msg00046.html

I guess this means my problem is unsolvable, short of a change in glibc.  :)  We are proceeding with the assumption that any non-negative-one return value of lseek64 indicates a successful seek.  I am proceeding to investigate if the seek really is successful by writing to my device in a controlled manner.
0
 
PistosChristouAuthor Commented:
Solved the problem.  I was just missing a #include <unistd.h>, and apparently, the compiler allowed it to compile anyway, presumably using function and type definitions from some other mysterious source.
0
 
sunnycoderCommented:
PAQ/Refund
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.