lseek64 - 8 gigabyte limit?

How can I lseek64 to 8 gigabytes or beyond?

I am opening my file (well, raw device) with open64, and with the O_LARGEFILE flag.  My lseek64's are being called with SEEK_SET.  I am using full-fledged off64_t's, not off_t's.

lseek64 calls seem to be alright up to an 8 gigabyte barrier, at which point, the position I try to seek to seems to be getting modulused by 8 gigabytes.  i.e. 8 gigs seeks to 0, 8 gigs + 1k seeks to 1k, and so on.  At least, that is what the return values of lseek64 tell me.

I have googled all over, and google-groups'ed for solutions, but any slightly relevant discussion is about the 2- or 4-gig limit of regular lseek.

This is on a Red Hat 7 hybrid, running a modified 2.4.18 kernel (with SMP), compiled with "cc (GCC) 3.2.2 20030222".  My program runs on a reiserfs filesystem.
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

u can check this link and see if u get any hint.
PistosChristouAuthor Commented:
shivsa: I've half-read, half-skimmed it, but it does not help my problem.
I never heard/encountred this milit for lseek64 ... may be because I never had to seek so far ...
If you are sure this is the problem, then a solution can be

while ( seek_length > 8 gb )
        lseek64 ( 8gb );
        seek_length = seek_length - 8gb;

lseek64 ( seek_length );  
Build an E-Commerce Site with Angular 5

Learn how to build an E-Commerce site with Angular 5, a JavaScript framework used by developers to build web, desktop, and mobile applications.

PistosChristouAuthor Commented:
sunnycoder: I will try incremental seeking as you suggest.  For the moment, I am doing:

off64_t position_desired = (some number > 8 gigs);
off64_t actual_position = lseek64( fd, position_desired, SEEK_SET );

and then (actual_position != position_position).
PistosChristouAuthor Commented:
Okay, incremental seeking doesn't help.  If I increment in 512 MB increments, it seeks fine at 512, 1024 and 1536 MB, but hitting 2 gigs causes it to return -2 gigs as a return value, leading me to believe that off64_t is being typed as a 32-bit signed integer, as opposed to the long long that it is supposed to be.
Dear  PistosChristou,

there are lots of projects goin in LARGEFILE support areas, and this area is not yet stable.
till now linux has good stable support for files upto 4GB and but not above than that.

please check the site for new developements and all.

good luck !
In that case, I am afraid you have not either used 64 interface consistently or you have not compiled the program with appropriate flags

you need to Compile your programs with "gcc -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE".
also make sure that you use the O_LARGEFILE flag with open to operate on large files
PistosChristouAuthor Commented:
sunnycoder: I have done a search through my source code, and every relevant open is an open64, and every lseek is an lseek64.  Each open has the O_LARGEFILE flag, and my compiler flags include: "-D_LARGEFILE64_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64".

shivsa: Thank you for the links.

In case it is of relevance, here is my glibc version:

$ /lib/
GNU C Library stable release version 2.3.2, by Roland McGrath et al.
Copyright (C) 2003 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
Compiled by GNU CC version 3.2.2 20030222 (Red Hat Linux 3.2.2-5).
Compiled on a Linux 2.4.20 system on 2003-02-27.
PistosChristouAuthor Commented:
Further possibly relevant information:

The program is being compiled on a different machine than the one it's being executed on.
>The program is being compiled on a different machine than the one it's being executed on.
try compiling and running on the same machine and post the results

> causes it to return -2 gigs as a return value,
is suspect ... I am not so sure about lseek64 but lseek returns -1 on failure and a positive value otherwise ... I believe lseek64 must not be too much different ....
PistosChristouAuthor Commented:
Compiling on the same machine doesn't bear any fruit.

> > causes it to return -2 gigs as a return value,
> is suspect ... I am not so sure about lseek64 but lseek returns -1 on failure and a positive value otherwise ... I believe lseek64 must not be too much different ....

Yes, I know, so this makes things rather, as you say, suspect.  Perhaps some high-order or low-order bits are being shifted, truncated, zeroed... who knows.  I've tried casting every relevant operand of every relevant operation, but it doesn't help.  The number being passed in is validly > 2 gigs, but the number being spit out is not.

Is it a possibility that my 64-bit calls and data types are being transformed at compile time to 32-bit versions, due to some header or library being missing or corrupt?
PistosChristouAuthor Commented:
Okay, someone else working on the project has come to the conclusion that the problem is a bug in glibc, wherein lseek64, though performing the seek correctly, does not return more than 32 bits.  A thread that we found:

I guess this means my problem is unsolvable, short of a change in glibc.  :)  We are proceeding with the assumption that any non-negative-one return value of lseek64 indicates a successful seek.  I am proceeding to investigate if the seek really is successful by writing to my device in a controlled manner.
PistosChristouAuthor Commented:
Solved the problem.  I was just missing a #include <unistd.h>, and apparently, the compiler allowed it to compile anyway, presumably using function and type definitions from some other mysterious source.
PAQed, with points refunded (200)

E-E Admin

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Linux OS Dev

From novice to tech pro — start learning today.