[Webinar] Streamline your web hosting managementRegister Today

x
?
Solved

urgent: make off_t 32 bits...

Posted on 2010-04-08
18
Medium Priority
?
298 Views
Last Modified: 2012-05-09
hi,

i am using off_t to store and retrieve file position..

however, i am storing unsigned ints (4 bytes) on file...

however, the off_t is 8 bytes...since i am using 32 bits, i thought off_t should be 4 bytes and not 8 bytes...it is just that i am manipulating the positions with the filepointer...having it behave like 64 bits is difficult...could i force it to be 32 bits....

thanks..
0
Comment
Question by:zizi21
  • 7
  • 4
  • 4
  • +2
18 Comments
 
LVL 33

Assisted Solution

by:phoffric
phoffric earned 400 total points
ID: 30160734
Not familiar with large 12MB files, so I may not be able to help.
My  initial review is that there are lseek64 and off64_t that are to be used for large files.
       http://www.delorie.com/gnu/docs/glibc/libc_240.html

The write (or pwrite) is defined as:
     ssize_t write(int fildes, const void *buf, size_t nbyte);
     ssize_t pwrite(int fildes, const void *buf, size_t nbyte, off_t offset);
Ref:  http://www.opengroup.org/onlinepubs/000095399/functions/write.html

>> however, i am storing unsigned int (4 bytes) on file...
Is this where you think you have a problem? Even though on your platform off_t is 8 bytes, I don't see off_t used in the write function. (It is in the pwrite function, so it appears that you will be able to do a write without having first to do an lseek or lseek64, if pwrite is available.)

Both writes do not seem to know about writing out 8 bytes to the file or for that matter 4 bytes. It only knows about writing out nbyte to the file. If you do a sizeof(size_t) do you get 4 or 8 bytes?

>> it is just that i am manipulating the positions with the file-pointer...having it behave like 64 bits is difficult...could i force it to be 32 bits
Well if you have a file-pointer of type off_t, and you are saying that it is 64 bits (presumably like an unsigned long long), then could you clarify what problem you are having. I'm not sure what kind of manipulation you are talking about since you are working with 64 bits. Doesn't your 64 bit type allow you to do the basic arithmetic + - * ?
0
 
LVL 33

Expert Comment

by:phoffric
ID: 30160848
Perhaps you could describe what kind of file-pointer manipulations you need to do, and show the code that is difficult to work with.

Also, what function are you using to retrieve the 64 bit file position?
0
 

Author Comment

by:zizi21
ID: 30161301
hi phoffric,

on the 32 bit platform, sizeof (off_t) is 8 bytes..
to make things easier for me, i am trying to make it work for small file (48 bytes) and then, i would try for large files..

i am writing 4 bytes of unsigned int to disk. there are times when i read the first 4 bytes (which is unsigned int) and then, i need to write another value on that position or on the next 4 bytes...


if off_t is 8 bytes, it is placing its pointer at the start of the second unsigned int value,but on the start of the third...
0
Will You Be GDPR Compliant by 5/28/2018?

GDPR? That's a regulation for the European Union. But, if you collect data from customers or employees within the EU, then you need to know about GDPR and make sure your organization is compliant by May 2018. Check out our preparation checklist to make sure you're on track today!

 

Author Comment

by:zizi21
ID: 30161330
if off_t is 8 bytes, it is not placing its pointer at the start of the second unsigned int value,but on the start of the third...
0
 

Author Comment

by:zizi21
ID: 30161406
for example, value in file is
 2 4
[4 bytes][4 bytes]
0
 
LVL 13

Expert Comment

by:Superdave
ID: 30161697
It won't hurt to store a 32-bit unsigned long quantity and cast it to 64 bits, as long as the file is under 4 GiB.  Or even signed long, and under 2 GiB.  I'm not sure that you need to though; I'd like to see what you're trying to do too.
0
 
LVL 13

Expert Comment

by:Superdave
ID: 30161987
What does the contents of the file have to do with the size of off_t?  Are you writing the offsets to the file for some reason?
0
 
LVL 33

Expert Comment

by:phoffric
ID: 30162182
Hi zizi21,

Are you doing anything different because you find that off_t is 64 bits? The offset, according to the lseek link, is just a "count of characters" so a 64 bit off_t should allow you to put in larger offsets and go beyond the 4GB limit. But the fact that it is 64 bits shouldn't make a difference, at least not for your small files.

I checked on my cygwin platform for the sizeof(off_t) and the sizeof(size_t). They are 8 and 4, respectively. So, maybe I can help. If you would like to post a full program (cut to a few lines in a main that builds) and runs, but produces the wrong data in the wrong offsets, I'll be happy to try to see if I can reproduce the problem on cygwin.
0
 
LVL 33

Expert Comment

by:phoffric
ID: 30165024
See you tomorrow if you still need help. Good luck!
0
 
LVL 40

Accepted Solution

by:
evilrix earned 1600 total points
ID: 30181097
@zizi21, you might find it useful to consider using the Related Question button :)
http://www.experts-exchange.com/help.jsp#hs=29&hi=414

Related questions
http://www.experts-exchange.com/Programming/Languages/C/Q_25772953.html
http://www.experts-exchange.com/Programming/Languages/C/Q_25773673.html
http://www.experts-exchange.com/Programming/Languages/C/Q_25774114.html

>> My  initial review is that there are lseek64 and off64_t that are to be used for large files.
If you have LFS enabled off_t is 64 bit
http://www.suse.de/~aj/linux_lfs.html

>> however, the off_t is 8 bytes...since i am using 32 bits
off_t represent the offset within the file, it is unrelated to the size of the data items being written. In other words, it is just a way to index into the file and the reason it's 64 bits with LFS enables is so you can address > 2GB files.

The fact LFS is enabled and that off_t is 64 bit when it is isn't relevant to writing small files or items smaller than 64 bit... you seem to be confusing yourself. You would write data in exactly the same was using exactly the same functions as if LFS was not enabled. All enabling LFS does (at compile time) is replace (or map) 32 bit file IO data types and functions their 64 bit versions. This happens under the hood and should be completely transparent to you. The only difference you should see if that you can create and manipulate files > 2GB. So, forget that LFS is enabled and just write this as a normal 32 bit piece of code.

If you are having a specific problem post the code here and I'll check if for you.

0
 

Author Comment

by:zizi21
ID: 30185975
sorry for the late reply..i had to rush for a class which i thought i could skip..

while(!feof(fp))
{

fread(&y,sizeof(unsigned int),1,fp);

}

i think, i have confused the 64 bit stuff with normal coding..so i have removed everything that i have added and just pretend it is for 32 bit..i am going to work on it now...i really appreciate of all your help into looking into this..
0
 
LVL 40

Expert Comment

by:evilrix
ID: 30186230
>>  i have added and just pretend it is for 32 bit
Good plan :)

I'll wait to hear from you.
0
 

Author Comment

by:zizi21
ID: 30189281
-->phoffric:
See you tomorrow if you still need help. Good luck!

thanks..

-->evilrix
'll wait to hear from you.

thanks...

thank you all so much and God bless you,
0
 
LVL 40

Expert Comment

by:evilrix
ID: 30189381
zizi21, you are most welcome. It's wonderful to assist someone who appreciates us. That means so much more than the points :)
0
 
LVL 39

Expert Comment

by:itsmeandnobodyelse
ID: 30191642
>>>> manipulating the positions with the filepointer...having it behave like 64 bits is difficult...could i force it to be 32 bits....

If you want to store file positions somewhere it firstly is a logical problem and not a technical one. Assume you know for sure that your files were all smaller than 4 GB. Then you *always* can store the position as 32bit integer even if ýou retrieved it as 64-bit integer. Simply cast it to UINT32 (what is a typedef of unsigned int on a 32bit platform) and store the byte position that way. If files got bigger you either could store 64-bit (or 48-bit) integers what would be enough for ever ... or if you were stuck on 32 bit for storing you could consider whether your file positions could be counted in records rather than in bytes. If for example you have 256 byte records and your positions always would/could point to the begin of such a record, you easily could store 'record numbers' for files of size 1000 GB with that approach.
0
 
LVL 40

Expert Comment

by:evilrix
ID: 30192119
The issue is that off_t can be either 32 bit or 64 bit, depending upon how it's compiled (if LFS is enabled). If you go casting from off_t to unsigned int without knowing if the file you are opening is > 2GB you, potentially, introduce a very hard to track down defect (trust me, I know... I spent a few days tracking down this exact type of bug a couple of months ago when another programmer made an assumption, much as this, on code built to handle large files).

Whilst I agree that your answer is, technically, correct... I'd very much recommend not doing this unless there is a compelling reason to do so.
0
 

Author Closing Comment

by:zizi21
ID: 31712601
thank you very much and God bless you all. .i really appreciate it.
0
 

Author Comment

by:zizi21
ID: 30239248
evilrix,

"zizi21, you are most welcome. It's wonderful to assist someone who appreciates us. That means so much more than the points :)"

thank you very much for your kind words...

0

Featured Post

Learn to develop an Android App

Want to increase your earning potential in 2018? Pad your resume with app building experience. Learn how with this hands-on course.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Summary: This tutorial covers some basics of pointer, pointer arithmetic and function pointer. What is a pointer: A pointer is a variable which holds an address. This address might be address of another variable/address of devices/address of fu…
This is a short and sweet, but (hopefully) to the point article. There seems to be some fundamental misunderstanding about the function prototype for the "main" function in C and C++, more specifically what type this function should return. I see so…
The goal of this video is to provide viewers with basic examples to understand and use pointers in the C programming language.
The goal of this video is to provide viewers with basic examples to understand opening and reading files in the C programming language.
Suggested Courses

608 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question