• C

C: function to determine freespace has issues with large amounts of freespace

Hello,
We use the C library function dos_getdiskfree() to determine if there is enough free space  on the hard drive. One example is in a utility that creates a bunch of tempfiles, whose size we can predict at runtime. If there is insufficient freespace, the program gracefully aborts.
------------------------------------------
unsigned _dos_getdiskfree( unsigned drive,
                        struct diskfree_t *diskspace );
struct diskfree_t {
        unsigned short total_clusters;
        unsigned short avail_clusters;
        unsigned short sectors_per_cluster;
        unsigned short bytes_per_sector;
};
------------------------------------------

We do the following:
return((long)diskspace.avail_clusters * (long)diskspace.sectors_per_cluster * (long)diskspace.bytes_per_sector);

Now, the problem. Because the unsigned long type in the 32-bit world is limited to 2^32, when the freespace EXCEEDS 2^32, the value wraps around. So, for example, if the free space is 2^32 + 1,000,000, the freespace is considered to be 1,000,000. Same thing for N* (2^32) + 1,000,000.

Does anyone know of a workaround for this. (e.g. is there a library function in the 32-bit world that uses either a 64-bit int or a floating point value to avoid this problem?

Thanks,
Steve
LVL 4
Stephen KairysTechnical Writer - ConsultantAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Ken ButtersCommented:
Sorry updating answer... misread your question..

Try this one:

GetDiskFreeSpace(), GetDiskFreeSpaceEx()
0
Stephen KairysTechnical Writer - ConsultantAuthor Commented:
OK, I tried this, but not only did I get a "no prototype" warning when compiling, but also  a undefinted reference when linking...

diskfree.c(56): Warning! W301: No prototype found for 'GetDiskFreeSpace'
diskfree.c(57): Warning! W301: No prototype found for 'GetDiskFreeSpaceEx'

Warning! W1028: GetDiskFreeSpace_ is an undefined reference
Warning! W1028: GetDiskFreeSpaceEx_ is an undefined reference

Sounds like maybe this function is not in my "C" libraries?
Thanks,
Steve
0
Stephen KairysTechnical Writer - ConsultantAuthor Commented:
OK, I tried something on my own.
I rewrote my local function to work with floats

float float_disk_free(drive)
char drive;
{

      struct diskfree_t diskspace;

      _dos_getdiskfree(drive, &diskspace);

      return( (float)diskspace.avail_clusters *
                            (float)diskspace.sectors_per_cluster *
                            (float)diskspace.bytes_per_sector);

}

When I do that, (on a drive with Per 8 sectors per cluster and 512 bytes per sector this would allow us  - I THINK - don't quote me on this - up to 17 terabytes based on the restriction that the available clusters is less than 2^32.  

Obviously, at some point this would need to be revisited as hard drives get larger but wondering if you think this sol'n is viable?
Thanks again.
Steve
0
Powerful Yet Easy-to-Use Network Monitoring

Identify excessive bandwidth utilization or unexpected application traffic with SolarWinds Bandwidth Analyzer Pack.

Ken ButtersCommented:
Yes definitely viable.

I remember reading a post on a 16bit issue, (similar to this one), and they mentioned to make sure not to use 16bit's (Shorts) variables to hold the response.

You are experiencing same issue except with 32bit vs 64 bit instead of 16bit vs. 32bit, but the concept is the same.
0
Stephen KairysTechnical Writer - ConsultantAuthor Commented:
Fair point. Thanks.
Now...what would be the format specifier for a 64-bit integer?
Thanks again.
-Steve
0
Ken ButtersCommented:
googling gave me this:

In Windows, it is "%I64d". In Linux and Solaris, it is "%lld".
0
Stephen KairysTechnical Writer - ConsultantAuthor Commented:
Thanks.
As it turns out, our compiler does not seem to support 64-bit intergers so I am going to go with the (float) solution. I will need to test this, which probably won't happen 'til next week so I'll leave the question open until I can verify this. Thanks for your help.
Steve

PS-Clarification: The complier error I got was:

testfree.c(57): Error! E1037: Type cast must be a scalar type
per
return( (signed_64)diskspace.avail_clusters * (signed_64)diskspace.sectors_per_cluster * (signed_64)diskspace.bytes_per_sector);

Thanks again.
0
sarabandeCommented:
the dos functions you were using are only supporting 32-bit.

you need to use GetDiskFreeSpace as mentioned by buttersk.

those functions are available via windows.h.  

the GetDiskFreeSpace has the following prototype:

BOOL WINAPI GetDiskFreeSpace(
  _In_   LPCTSTR lpRootPathName,
  _Out_  LPDWORD lpSectorsPerCluster,
  _Out_  LPDWORD lpBytesPerSector,
  _Out_  LPDWORD lpNumberOfFreeClusters,
  _Out_  LPDWORD lpTotalNumberOfClusters
);

Open in new window


the LPCTSTR would evaluate to const char * if your project doesn't uses UNICODE character set (what seems to be sure as you were not using WINAPI).

The LPDWORD are pointers to 32-bit unsigned integer. as big disks have a default cluster size of 64k, the dword would take up to 64k*4 GB  what is 256 TB. that should be enough even for big environments.

for calculation of free bytes you would use 64 bit type:

__int64 nfb = 0, nf, nbps, nspc;
DWORD df = 0, dbps = 0, dspc = 0;

if (GetDiskFreeSpace("C:\\", &dspc, &dbps, &df, NULL) )
{
      nf = df;
      nbps = dbps;
      nspc = dspc;
      nfb = nf * nbps * nspc;
} 
if (nfb < (__int64)bytes_required)
{
     ....

Open in new window


if you want to avoid 64 bit calculations you could use GetDiskFreeSpaceEx which already uses 64bit integer type, however it is not __int64 but structure ULARGE_INTEGER which is not quite handy with 32-bit members lowPart and highPart. there are conversion functions and you could cast a pointer to the struct to a pointer of __int64 but actually I don't like those types and the fussiness involved by them.

Sara
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Stephen KairysTechnical Writer - ConsultantAuthor Commented:
Sara,
Thanks. I'll have to look at this in more detail when I have more time...two other projects have taken me away from this for now. That said, my compiler does not seem to think that signed_64 is a scalar type.

Also, you said that big disks have a default clutersize of 64K. On my disk, the old dos_getdiskfree() function returns:

sectors per cluster: 8  bytes per sector 512
which would be a cluster size of 4,096 (4K, I guess). Per the 64K clustersize, do you know what the sectors per cluster and bytes per sector would be?

Thanks
Steve
0
sarabandeCommented:
the default cluster size changes depending on the size of disks. this is true mainly for ntfs file system. for fat and fat32 it is different. but it is also possible to configure a different cluster size (though it would not happen for almost all systems). generally, if the required file size (the value you have check against) is a 32-bit integer and therefore less than max 2^32 (4GB) the GetDiskFreeSize would be good enough even if the user has a partition with 2, 3 or 4 TB in size.

note, the dos functions where made to a time where disk sizes count in mega bytes. the where upper boundaries of 2 GB disks (2^31) which only could be handled by bios enhancements (tricks).

Sara
0
Stephen KairysTechnical Writer - ConsultantAuthor Commented:
Sara,
I pasted in your code and it looks promising....per my output...

nfb= 45742166016
this is in a test program and need to see how it works in my utility.

Btw, you use the type DWORD.  Is that the same as a 32-bit interger? If so, why not just declare as a long or (in the 32-bit world) an int?

Thanks again,
Steve
0
sarabandeCommented:
a DWORD is defined in the windef.h as 'unsigned int'. it therefore evaluates from 0 to 2^32-1.

note, also most 64-bit platforms have int and long defined as 32-bit integers.

i used DWORD cause the WINAPI (windows application programming interface) uses these types. you may replace them by 'unsigned int' or 'unsigned long' if you like better.

Sara
0
Stephen KairysTechnical Writer - ConsultantAuthor Commented:
Sara,
I am on a 32-bit platform. Do all of the above remarks still apply?  
Thanks
Steve
0
sarabandeCommented:
yes. the api GetDiskFreeSize and GetDiskFreeSizeEx works for any windows platform up to windows 8.1.

Sara
0
Stephen KairysTechnical Writer - ConsultantAuthor Commented:
Pardon my ignorance but 8.1 being the latest? Can we assume it would work for generations of windows beyond 8.1?
Thanks
0
sarabandeCommented:
yes and no.

yes, because an api used by millions of programs cannot be changed but only enhanced. there would be a new GetDiskFreeSpaceExEx that takes __int64 types and a partition name rather than a drive letter but the old functions will still work.

no, because disks (exactly partitions and drives) and disk free space might be obsolete for future versions of windows if the trend towards cloud computing and away from local storage goes on. but as there are a lot of people which were dependent on the current technology i would not worry about that.

Sara
0
Stephen KairysTechnical Writer - ConsultantAuthor Commented:
OK that's fine. Final question; What should determine if I use GetDiskFreeSpace() or GetDiskFreeSpaceEx()?
Tks.
0
sarabandeCommented:
you can go with either call. if you feel good with GetDiskFreeSpace and the 32-bit calculations, it should work some years.

the GetDiskFreeSpaceEx already uses 64-bit (what is better) but uses strange structure type ULARGE_INTEGER.

you could convert from ULARGE_INTEGER to __int64 by


ULARGE_INTEGER uli = { 0 };
GetDiskFreeSpaceEx(...., &uli, ...);
 
__int64 i64 = *(__int64*)(&uli); // cast pointer to struct to int64 pointer and dereference 

Open in new window


it is ugly code but if you encapsulate it into a function, the GetDiskFreeSpaceEx might be the better call on mid-term or long-term.

Sara
0
Stephen KairysTechnical Writer - ConsultantAuthor Commented:
>>You can go with either call. if you feel good with GetDiskFreeSpace and the 32-bit >>calculations, it should work some years.
32-bit calculations? I thought the whole point of using GetDiskFreeSpace() was to avoid the 2^32-1 (apx. 4 billion)  limit imposed by 32-bit integers.
Thanks.
0
sarabandeCommented:
yes, but the whole efforts we did was to use 32-bit input properly to not getting an overflow.

the Ex function is already 64-bit what needs much less carefulness.

Sara
0
Stephen KairysTechnical Writer - ConsultantAuthor Commented:
OK Thank you.
0
Stephen KairysTechnical Writer - ConsultantAuthor Commented:
Update....
We tried this on a win2000 machine (actually a virtual machine) and the value given was off for the total freespace.

The acutal value is about 88,xxx,xxx,xxx but under w2000 it was about 92,xxx,xxx,xxx.
Is this behavior expected?
Thanks,
Steve
0
sarabandeCommented:
the difference looks like the differences that occur when 1MB was counted as 1 million rather than 10^6 == 1048576. you may divide the w2k number by 1.0485576 and check whether you get the other result.

I don't know for sure but I mean to remember that windows os before xp got their disk info from bios while later windows would get from disk itself, what could be an explanation for the difference.

Sara
0
Stephen KairysTechnical Writer - ConsultantAuthor Commented:
Well, actually I mis-read the values from last night's test.
Actual free space: 88,287,141,888
Incorrect value: 9,255,321,600 (I misread as 92,xxx,xxx,xxx)

So maybe its the bios vs. disk thing?

Thanks
Steve
0
sarabandeCommented:
it is the bios but another reason. the bios limits for disks could not handle actual multi-gigabyte disks because the parameters for sizing have maximums that were too small.

see http://www.tldp.org/HOWTO/Large-Disk-HOWTO-4.html

so, the sizes calculated from the bios values sectors, heads, cylinders necessarily were wrong. that doesn't matter as long as the installed drivers were able to handle bigger sizes by reading info directly from disk. that also doesn't matter as long as by using additional bios parameters (LBA), functions can correct the results. as long as only disk info is wrong, it also may be acceptable for most applications. but for win2k I don't know whether it could handle the 137 GB limit of 2002 at all, what would mean that on a 512GB disk you could not create partitions greater than 137GB.

Sara
0
Stephen KairysTechnical Writer - ConsultantAuthor Commented:
Sara and Buttersk-
Thank you for your help. Sara giviing you the majority of the points per your more detailed sol'n and long dialogue we had.

Per my test program your (Sara)'s solution appears to do what I need; however, b/c of some issues with my application dealing with WINAPI's and customers still on w2000, I can't (for now at least ) go forward with it.

So I am going to use floats or doubles. I realize I may lose a bit of precision; however, I can live with that since I don't need to be exact.

Thanks again,
Steve
0
Stephen KairysTechnical Writer - ConsultantAuthor Commented:
Follow-up:
So now that i was resigned to using dos_getdiskfree, I was poking around, and noticed that there is also a function getdiskfree().  In the documentation I have, I can't seem to see any major differences aside from:

For dos_getdiskfree()
"Systems:
DOS, Windows, Win386, Win32, OS/2 1.x(all), OS/2-32, DOS/PM"

For get_diskfree()
"Systems:
DOS, Windows, Win386, Win32, OS/2 1.x(all), OS/2-32"

So are these two functions essentially the same?
Thanks,
Steve
0
sarabandeCommented:
yes. also GetDiskFreeSpace firstly added by Win NT 3.0 returned the same (bios) values.

the only difference is that newer windows systems would make some corrections to the values if bios limits or 32-bit limits were violated. also the bios calls (interrupts) would return corrected values for newer systems (for example LBA).

so it is always a combination of bios and os which either returns correct values or would fail if the disks were too big.

if you are happy with the dos_getdiskfree it is ok. but you should not cast to float or double but to __int64 what has the same effect that the multiplication would not go beyond 32-bit but gives a correct integer result.

Sara
0
Stephen KairysTechnical Writer - ConsultantAuthor Commented:
>>yes. also GetDiskFreeSpace firstly added by Win NT 3.0 returned the same (bios) values.
>>the only difference is that newer windows systems would make some corrections to the values if bios limits or 32-bit limits were violated. also the bios calls (interrupts)
>>would return corrected values for newer systems (for example LBA).

So ARE you saying that dos_getdiskfree() and getdiskfree() should be equivalent?

>>so it is always a combination of bios and os which either returns correct values or would >>fail if the disks were too big.
Meaining that I could still get in trouble with these functions as hard drives get bigger, even with doing the __int64 thing? I'd rather not have to revisit this issue down the road.

>>if you are happy with the dos_getdiskfree it is ok. but you should not cast to
>>  float or double but to __int64 what has the same effect that the >>multiplication would not go beyond 32-bit but gives a correct integer result.

Reason?  If you're concerned about precision issues with float/double that does not matter to us...we're just need an aproxmiate value.
0
sarabandeCommented:
dos_getdiskfree() and getdiskfree() and GetDiskFreeSpace return the same figures deduced from bios parameters and/or additional info.

depending on the bios and the operation system it could be different results and different limits where the results might be wrong.

we're just need an aproxmiate value
yes, i understand. it is only that i don't like casts, especially if they lead to loss of information. it doesn't matter in that case and might matter in another case. with little efforts you could use 64-bit integers  for calculation. if you then divide by (1024*1024) == 1MB, you could return the result as 32-bit (unsigned) integer.

Sara
0
Stephen KairysTechnical Writer - ConsultantAuthor Commented:
So why divide by 1MB then. I am fine with my function returning an __int64 value. I tried it on several hard drives, including a network drive with freepace of
4,389,540,007,936 bytes
and the value it returned was exact.
I was about to close this out but then I realized that the number of clusters on this drive
is VERY close to 2^32:

4,286,660,176
vs.
4,294,967,296

And since the diskfree_t structure is
struct _diskfree_t {
        unsigned total_clusters;
        unsigned avail_clusters;
        unsigned sectors_per_cluster;
        unsigned bytes_per_sector;
};

I"m out of luck, right?

So, it sounded like I may have to bite the bullet and go back to GetDiskFreeSpace() which I hoped has no such concerns. I was feeling pretty good about that until I realized that
the individual parameters that you have multiply to arrive at the total free space are will 32-bit integers so once again, if I drive has > (2^32-1) clusters free I'm out of luck this way too, right?

At this point, I'm thinking I need to go out of the box, have my program call "DIR", direct the output to a text file and parse the line that displays the # of bytes free to derive the value that way.

Thanks
-Steve





 As far as my w2000 concerns, I"m almost the point of saying it's the 21st Century :) , and I can't worry about supporting old technology, esp. since
the number of installations we have whose servers are w2000 are few and dwindling.

Thanks
0
Stephen KairysTechnical Writer - ConsultantAuthor Commented:
Sara,
I think I got it. I went back to your suggestion per GetDiskFreeSpaceEx(() and it turned out to be pretty painless. This really looks to be the most promising sol'n, and I've decided I don't care if it does not work on win2000.

      
void ULargeDiskFree(char DriveLetter)
{
 ULARGE_INTEGER freeSpace;
  ULARGE_INTEGER totalSpace;
  ULARGE_INTEGER totalFreeSpace;

  char DriveSpec[10];  // e.g. 'C' --> "C:\"
__int64 i64TotalFreeSpace;


  GetDiskFreeSpaceEx(DriveSpec, &freeSpace, &totalSpace, &totalFreeSpace );

 
  i64TotalFreeSpace = *(__int64*)(&totalFreeSpace); // cast pointer to struct to int64 pointer and dereference 

printf("casted totalspace = %I64d\n", i64TotalFreeSpace);

} // ULargeDiskFree()

Open in new window


 

Thank you again. If I could re-open this question and give you another 500 points - or  more - I would. :)

Steve
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
C

From novice to tech pro — start learning today.