• C

urgent: make off_t 32 bits...

hi,

i am using off_t to store and retrieve file position..

however, i am storing unsigned ints (4 bytes) on file...

however, the off_t is 8 bytes...since i am using 32 bits, i thought off_t should be 4 bytes and not 8 bytes...it is just that i am manipulating the positions with the filepointer...having it behave like 64 bits is difficult...could i force it to be 32 bits....

thanks..
zizi21Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

phoffricCommented:
Not familiar with large 12MB files, so I may not be able to help.
My  initial review is that there are lseek64 and off64_t that are to be used for large files.
       http://www.delorie.com/gnu/docs/glibc/libc_240.html

The write (or pwrite) is defined as:
     ssize_t write(int fildes, const void *buf, size_t nbyte);
     ssize_t pwrite(int fildes, const void *buf, size_t nbyte, off_t offset);
Ref:  http://www.opengroup.org/onlinepubs/000095399/functions/write.html

>> however, i am storing unsigned int (4 bytes) on file...
Is this where you think you have a problem? Even though on your platform off_t is 8 bytes, I don't see off_t used in the write function. (It is in the pwrite function, so it appears that you will be able to do a write without having first to do an lseek or lseek64, if pwrite is available.)

Both writes do not seem to know about writing out 8 bytes to the file or for that matter 4 bytes. It only knows about writing out nbyte to the file. If you do a sizeof(size_t) do you get 4 or 8 bytes?

>> it is just that i am manipulating the positions with the file-pointer...having it behave like 64 bits is difficult...could i force it to be 32 bits
Well if you have a file-pointer of type off_t, and you are saying that it is 64 bits (presumably like an unsigned long long), then could you clarify what problem you are having. I'm not sure what kind of manipulation you are talking about since you are working with 64 bits. Doesn't your 64 bit type allow you to do the basic arithmetic + - * ?
0
phoffricCommented:
Perhaps you could describe what kind of file-pointer manipulations you need to do, and show the code that is difficult to work with.

Also, what function are you using to retrieve the 64 bit file position?
0
zizi21Author Commented:
hi phoffric,

on the 32 bit platform, sizeof (off_t) is 8 bytes..
to make things easier for me, i am trying to make it work for small file (48 bytes) and then, i would try for large files..

i am writing 4 bytes of unsigned int to disk. there are times when i read the first 4 bytes (which is unsigned int) and then, i need to write another value on that position or on the next 4 bytes...


if off_t is 8 bytes, it is placing its pointer at the start of the second unsigned int value,but on the start of the third...
0
Powerful Yet Easy-to-Use Network Monitoring

Identify excessive bandwidth utilization or unexpected application traffic with SolarWinds Bandwidth Analyzer Pack.

zizi21Author Commented:
if off_t is 8 bytes, it is not placing its pointer at the start of the second unsigned int value,but on the start of the third...
0
zizi21Author Commented:
for example, value in file is
 2 4
[4 bytes][4 bytes]
0
SuperdaveCommented:
It won't hurt to store a 32-bit unsigned long quantity and cast it to 64 bits, as long as the file is under 4 GiB.  Or even signed long, and under 2 GiB.  I'm not sure that you need to though; I'd like to see what you're trying to do too.
0
SuperdaveCommented:
What does the contents of the file have to do with the size of off_t?  Are you writing the offsets to the file for some reason?
0
phoffricCommented:
Hi zizi21,

Are you doing anything different because you find that off_t is 64 bits? The offset, according to the lseek link, is just a "count of characters" so a 64 bit off_t should allow you to put in larger offsets and go beyond the 4GB limit. But the fact that it is 64 bits shouldn't make a difference, at least not for your small files.

I checked on my cygwin platform for the sizeof(off_t) and the sizeof(size_t). They are 8 and 4, respectively. So, maybe I can help. If you would like to post a full program (cut to a few lines in a main that builds) and runs, but produces the wrong data in the wrong offsets, I'll be happy to try to see if I can reproduce the problem on cygwin.
0
phoffricCommented:
See you tomorrow if you still need help. Good luck!
0
evilrixSenior Software Engineer (Avast)Commented:
@zizi21, you might find it useful to consider using the Related Question button :)
http://www.experts-exchange.com/help.jsp#hs=29&hi=414

Related questions
http://www.experts-exchange.com/Programming/Languages/C/Q_25772953.html
http://www.experts-exchange.com/Programming/Languages/C/Q_25773673.html
http://www.experts-exchange.com/Programming/Languages/C/Q_25774114.html

>> My  initial review is that there are lseek64 and off64_t that are to be used for large files.
If you have LFS enabled off_t is 64 bit
http://www.suse.de/~aj/linux_lfs.html

>> however, the off_t is 8 bytes...since i am using 32 bits
off_t represent the offset within the file, it is unrelated to the size of the data items being written. In other words, it is just a way to index into the file and the reason it's 64 bits with LFS enables is so you can address > 2GB files.

The fact LFS is enabled and that off_t is 64 bit when it is isn't relevant to writing small files or items smaller than 64 bit... you seem to be confusing yourself. You would write data in exactly the same was using exactly the same functions as if LFS was not enabled. All enabling LFS does (at compile time) is replace (or map) 32 bit file IO data types and functions their 64 bit versions. This happens under the hood and should be completely transparent to you. The only difference you should see if that you can create and manipulate files > 2GB. So, forget that LFS is enabled and just write this as a normal 32 bit piece of code.

If you are having a specific problem post the code here and I'll check if for you.

0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
zizi21Author Commented:
sorry for the late reply..i had to rush for a class which i thought i could skip..

while(!feof(fp))
{

fread(&y,sizeof(unsigned int),1,fp);

}

i think, i have confused the 64 bit stuff with normal coding..so i have removed everything that i have added and just pretend it is for 32 bit..i am going to work on it now...i really appreciate of all your help into looking into this..
0
evilrixSenior Software Engineer (Avast)Commented:
>>  i have added and just pretend it is for 32 bit
Good plan :)

I'll wait to hear from you.
0
zizi21Author Commented:
-->phoffric:
See you tomorrow if you still need help. Good luck!

thanks..

-->evilrix
'll wait to hear from you.

thanks...

thank you all so much and God bless you,
0
evilrixSenior Software Engineer (Avast)Commented:
zizi21, you are most welcome. It's wonderful to assist someone who appreciates us. That means so much more than the points :)
0
itsmeandnobodyelseCommented:
>>>> manipulating the positions with the filepointer...having it behave like 64 bits is difficult...could i force it to be 32 bits....

If you want to store file positions somewhere it firstly is a logical problem and not a technical one. Assume you know for sure that your files were all smaller than 4 GB. Then you *always* can store the position as 32bit integer even if ýou retrieved it as 64-bit integer. Simply cast it to UINT32 (what is a typedef of unsigned int on a 32bit platform) and store the byte position that way. If files got bigger you either could store 64-bit (or 48-bit) integers what would be enough for ever ... or if you were stuck on 32 bit for storing you could consider whether your file positions could be counted in records rather than in bytes. If for example you have 256 byte records and your positions always would/could point to the begin of such a record, you easily could store 'record numbers' for files of size 1000 GB with that approach.
0
evilrixSenior Software Engineer (Avast)Commented:
The issue is that off_t can be either 32 bit or 64 bit, depending upon how it's compiled (if LFS is enabled). If you go casting from off_t to unsigned int without knowing if the file you are opening is > 2GB you, potentially, introduce a very hard to track down defect (trust me, I know... I spent a few days tracking down this exact type of bug a couple of months ago when another programmer made an assumption, much as this, on code built to handle large files).

Whilst I agree that your answer is, technically, correct... I'd very much recommend not doing this unless there is a compelling reason to do so.
0
zizi21Author Commented:
thank you very much and God bless you all. .i really appreciate it.
0
zizi21Author Commented:
evilrix,

"zizi21, you are most welcome. It's wonderful to assist someone who appreciates us. That means so much more than the points :)"

thank you very much for your kind words...

0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
C

From novice to tech pro — start learning today.