Solved

MALLOC > 64K

Posted on 2003-11-23
47
1,106 Views
Last Modified: 2007-12-19
Hi to All!

I want to allocate memory in pure C code (no OOP) as maximum as possible i have 128MB ram in pc but i can only occupy 64k max,

I am using Borland C v 3.1 Compiler under windows 98,

and if it is not possible in Borland C then there will be some other compiler out there, just give a link to download or any thing and code too,
                                                  BUT
again C (pure C) compilers what i want, as i donot want to change my exiting C code.
0
Comment
Question by:meet_zorrer
  • 14
  • 13
  • 12
  • +3
47 Comments
 
LVL 45

Expert Comment

by:sunnycoder
Comment Utility
Hi meet_zorrer,

I guess any windows/linux based compiler should do it for you ...

you can either use VC++ or gcc .... VC++ is ofcourse M$ product and gcc port can be downloaded from gcc.gnu.org

Cheers!
Sunny:o)
0
 
LVL 4

Author Comment

by:meet_zorrer
Comment Utility
Is GCC Pure C compiler or OOP only or both?

and what about the code???

PS: i donot want VC++, i already have in my PC
0
 
LVL 45

Expert Comment

by:sunnycoder
Comment Utility
gcc compiles C as well as C++ ... I think front ends for fortran and pascal are also being developed (or is it that they have been developed ) ...

to ensure that it compiles pure ANSI C, compile your .c files with -pedantic flag
0
 
LVL 4

Author Comment

by:meet_zorrer
Comment Utility
there are lot of versions of gcc what sould i use?? the latest one or some other stable version?
0
 
LVL 45

Expert Comment

by:sunnycoder
Comment Utility
the most recent stable version will be a good choice ... i am using 2.96 and works fine for me (note I am on Linux)
0
 
LVL 4

Author Comment

by:meet_zorrer
Comment Utility
i read here and there on gcc site but there is no binaries available for the windows but the cygwin project from http://sources.redhat.com/cygwin/ so i thing we are goin off track now to the question , as the gcc is not for windows }:-(
0
 
LVL 45

Expert Comment

by:sunnycoder
Comment Utility
gcc was originally built for linux but it has been ported to a number of platforms ... cygwin is linux simulator for windows ...
I have used VC++ for sometime in past but I like gcc better ... I am not sure if installing gcc on win requires you to have cygwin too ...

Wait for some windows guru to comment on this
0
 
LVL 22

Expert Comment

by:grg99
Comment Utility
If you need to use more than 64K, you'll need a compiler that can generate 32-bit protected-mode code.

I'd suggest Watcom C, as it's open source, free, and can do almost any memory model you might need.

www.openwatcom.org  I think is the place.

0
 
LVL 5

Expert Comment

by:g0rath
Comment Utility
LCC-Win32 is also a free C compiler that runs native under windows

http://www.cs.virginia.edu/~lcc-win32/
0
 
LVL 45

Assisted Solution

by:Kdo
Kdo earned 40 total points
Comment Utility

Hi meet_zorrer,

Borland's Turbo C (which is what you are using) is a 16 bit compiler.  This means that it has 16-bit addressing.  The maximum value that you can keep in 16 bits is 65535, hence you have a 64K limitation.

Granted, you said that you don't want to change your C code, but to get more than 64K you're going to have to move off of the 16-bit platform (using one of the 32-bit C compilers that has been mentioned) or make a small change to the code.

Borland's memory management routines have a faralloc() function that allows you to reserved more than 64K in a single block.  The function is a direct replacement for malloc(), except that the argument is a long int so values greater than 64K can be accomodated.

But be cautioned -- memory addressing is still a 16-bit operation.  If you faralloc() a buffer of (for example) 80K you'll need to play some tricks to get to the memory addresses above 64K.  The following code doesn't do what you want:

char *Buffer, *Ptr;

Buffer = faralloc(80000);
Ptr = Buffer + 70000;

Because memory is 16-bit, only the lower 16 bits of the value 70000 will be added to the pointer.  Ptr will actually point to Buffer + 4464.  (4464 is modulo (70000, 65536).)

But you can "work around" this quite effectively.

Buffer = faralloc (80000);
Ptr = Buffer + 32768;       /*  32768 is 65536/2  */
Ptr = Buffer + 32768;

Ptr now points to the first byte in the region above 65536.


Good Luck,
Kent
0
 
LVL 17

Expert Comment

by:rstaveley
Comment Utility
farmalloc returns far pointers, which can theoretically address any part of your 1M DOS (= 24 bit) memory address space. You get the same pointers returned from malloc if you are using a medium or large memory model, but you do not need the far keyword, because data pointers are implicitly far.

These 32-bit pointers are cranky, as indicated by Kent, because they have 16-bits for the segment and 16-bits for the offset. Far pointer arithmetic leaves the segment unchanged and only applies changes the offset, because who in their right mind would need more than 64K ;-), and it is generally optimal to work that way. Mostly far pointers are used to point to < 64K chunks of memory, but by having far pointers you don't have to have the sum of those chunks totalling < 64K.

If you want to address a chunk of memory which is > 64K, you should use a huge pointer. Huge pointers automatically do normalisation for you when you do pointer arithmetic... or rather the compiler adds code to normalise huge pointers. "Nomalising" means that you put as much of the 24 bits address into the segment as possible and have only the least significant nibble of the offset left with significance.

e.g. 1000:1234 is normalised to 1123:0004, which points to the same address, but leaves more room in the offset to do arithmetic.

I believe that you can directly index element 65536 of a char array pointed to by a huge pointer, but... it seems like a long time since I did work with DOS.

        char huge *chptr = (char huge*)farmalloc(80000);
        char far* cfptr = &chptr[65536]; /* I think that this works OK... but you should check :-) */
0
 
LVL 45

Expert Comment

by:Kdo
Comment Utility

Hi rstavely,

faralloc(), farmalloc() -- not sure if that was a slip of the fingers of a slip of the mind.  :)  Thanks for the correction.

Having huge pointers does make for certain niceties, but you still cant "directly" address more than 64K.  16 bits is 16 bits and address calculations are performed with 16 bit math.  (At least I don't recall the generated code jumping through hoops to do 24/32-bit math for huge pointers.)


Kent
0
 
LVL 22

Expert Comment

by:grg99
Comment Utility
The farmalloc() functions are fine, but they only get you access to DOS memory, which usually tops out around 600K.

Well, actually, I lie, there are ways to access more memory, but they're not of general use, they require a lot of calls to copy blocks
back and forth, and it's not only slow, it's a pain to use.

To get access to ALL your memory, you will need to make the jump to protected mode, either 16 or 32 bit varieties.

0
 
LVL 17

Expert Comment

by:rstaveley
Comment Utility
I've just dug up an old Borland compiler (Turbo C version 1 !!), and I can see that it does indeed do kosher pointer arithmetic in the example I gave.

Here's the code snippet:
--------8<--------
      char huge *chptr = (char huge*)farmalloc(80000);
      char far* cfptr = &chptr[65536]; /* I think that this works OK... but you should check :-) */
      *cfptr = 1;
--------8<--------

Here's the generated assembler code:
--------8<--------
      mov      dx,1
      mov      ax,14464
      push      dx
      push      ax
      call      near ptr _farmalloc
      pop      cx
      pop      cx
      mov      word ptr [bp-6],dx
      mov      word ptr [bp-8],ax
; Line 6
      mov      dx,word ptr [bp-6]
      mov      ax,word ptr [bp-8]
      mov      cx,1                               ; <- Here's it putting 0x10000 into cx:bx
      xor      bx,bx
      call      far ptr padd@                ; <- ..to put the correct address into dx:ax
      mov      word ptr [bp-2],dx
      mov      word ptr [bp-4],ax
      les      bx,dword ptr [bp-4]
      mov      byte ptr es:[bx],1           ; <- Here's the assignment
--------8<--------
The pointer arithmetic is done by an internal function padd@.

It does jump through some hoops, but isn't it great *not* to have to work with 24 bit code nowadays? :-)

Incidentally, none of this will address the 128MB, meet_zorrer. Virtual 86 DOS boxes in Windows '98 will only give you 640K to play with. You really ought to get a copy of GCC as previously mentioned - see http://www.mingw.org/
0
 
LVL 45

Expert Comment

by:Kdo
Comment Utility

Cool.  You learn something new (or old... very very old....) every day.  :)

BTW, I too, have distros for Turbo C 1, 2, and 3.

Kent
0
 
LVL 17

Expert Comment

by:rstaveley
Comment Utility
Remember how Turbo C 1 could fit on a bootable 720K floppy with space for PKZip, QEdit and an obligatory boot sector virus? There was no need for a lap-top when visiting installations with that in your shirt pocket! Makes me go all misty-eyed ;-)
0
 
LVL 45

Expert Comment

by:Kdo
Comment Utility
All of my Turbo C distributions came on 5 inch floppies.  (Am I the only bithead in the world that calls them "5 inch" floppies instead of "5 1/4 inch"?  If anyone ever markets a REAL 5-inch floppy I'll change with the times.)  IIRC, Turbo C 3.x is on 3 floppies plus Turbo Debugger on 2 more and Turbo Assembler on another.

Quite a difference between those and C++ Builder's 100MB+ install file.  :)

0
 
LVL 4

Author Comment

by:meet_zorrer
Comment Utility
All guys out there Thanks,
                           BUT

my problem is still there and even becoming less it gets up as now i have to setup the GCC and CYGWIN/ or learn new compilers like WahtCom etc...

what is the best round about if i have to stick to Borland C only , i am using Double link lists and memory goes off after about 10 to 15 nodes and the size of node is about 2 KB

so what sould be my new Plan:

1) Use the Random/Binary File handling forget about Memory
2) Use new compiler
3) or Donot use the Datastructure. use the Borland C as a 1 month old programer does it


ok Guys i am starting to get worried now! :-?
0
 
LVL 17

Expert Comment

by:rstaveley
Comment Utility
If ~200KB is enough for you (allowing for a ~300K program size), use farmalloc. If it isn't, you're going to have to bite the bullet.
0
 
LVL 45

Expert Comment

by:sunnycoder
Comment Utility
Unless your job or development environment revolves around 16 bit compilers, I would suggest move on to 32 bit compilers as they are the ones in most widespread use (like it or not)
0
 
LVL 4

Author Comment

by:meet_zorrer
Comment Utility
well i donot like it  }:-?
0
 
LVL 17

Expert Comment

by:rstaveley
Comment Utility
If you are loading a relatively large array of relatively small structs into RAM, you might consider reading and writing to a database.

e.g.

    static struct YOURSTRUCT current_record;
    int current_record_recnum; /* Record number of current record */
    int current_record_modified; /* Non-zero if the current record has been modified */
    FILE *file; /* Read/write binary file - the database - i.e. file = fopen("database.dat",r+b); */

    int set_current_record(int recnum)
    {
        /* Write current record to file, if it has been modified */
        if (current_record_modified) {
                if (fseek(file,sizeof(struct YOURSTRUCT)*current_recnum,SEEK_SET) != 0)
                        return 0;
                if (fwrite(&current_record,sizeof(struct YOURSTRUCT),1,file) == 1)
                        current_record_modified = 0;
        }

        /* Read record recnum from the file into the current record */
        if (current_record_recnum != recnum) {
                if (fseek(file,sizeof(struct YOURSTRUCT)*recnum,SEEK_SET) != 0)
                        return 0;
                if (fread(&current_record,sizeof(struct YOURSTRUCT),1,file) == 1)
                        current_record_modified = 0;
                else
                        return 0;
        }
        return 1;
    }

Do remember to use a medium or large memory model to ensure that your total data isn't confined to a 64K data segment.
0
 
LVL 4

Author Comment

by:meet_zorrer
Comment Utility
ok rstaveley! if your solution works then i will be a happy man again, I try it tonight so points gona be yours as i just want to get round about my problem!
0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 
LVL 45

Expert Comment

by:Kdo
Comment Utility

Hey -- rstavely....


> I've just dug up an old Borland compiler (Turbo C version 1 !!), and I can see that it does indeed do kosher pointer arithmetic in the example I gave.

> Here's the code snippet:
> --------8<--------
>      char huge *chptr = (char huge*)farmalloc(80000);
>      char far* cfptr = &chptr[65536]; /* I think that this works OK... but you should check :-) */
>      *cfptr = 1;
> --------8<--------

Do I smell a fast one being pulled here?  :)  TurboC should throw a compilation error when you try to index an array by a 17 bit value ????

char far* cfptr = chptr +65535;  /*  This ??might?? work  */


Kent

0
 
LVL 17

Accepted Solution

by:
rstaveley earned 60 total points
Comment Utility
17 bit??

The generated assembler code was indeed generated. You can see CX:BX being used to pass the long index.

Indexing the huge pointer works with a long, but indexing the far pointer doesn't:
--------8<--------
#include <stdio.h>
#include <alloc.h>

main()
{
      char huge *chptr = (char huge*)farmalloc(80000);
      char far* cfptr = (char far*)&chptr[65537];
      char far* cfptr2 = (char far*)&chptr[0];
      char far* cfptr3 = &cfptr2[65537];

      *cfptr = 123;
      printf("It is as easy as %d when you index the huge pointer\n",(int)chptr[65537]);
      printf("Huge pointer is %Fp\n",chptr);
      printf("Far pointer pointing to element 0x10001 is %Fp\n\n",cfptr);

      *cfptr3 = 0;
      printf("There are %d problems when you index the far pointer\n",(int)chptr[65537]);
      printf("Huge pointer is %Fp\n",chptr);
      printf("Far pointer used for indexing is %Fp\n",cfptr2);
      printf("Far pointer pointing to element 0x10001 is %Fp\n\n",cfptr3);
}
--------8<--------

Gets...
--------8<--------
It is as easy as 123 when you index the huge pointer
Huge pointer is 193E:0004
Far pointer pointing to element 0x10001 is 293E:0005

There are 123 problems when you index the far pointer
Huge pointer is 193E:0004
Far pointer used for indexing is 193E:0004
Far pointer pointing to element 0x10001 is 193E:0005
--------8<--------

The huge pointer indexed by a long returns a normalised pointer. The far pointer indexed by a long... is only indexed by the least significant 16-bits of the long.
0
 
LVL 17

Expert Comment

by:rstaveley
Comment Utility
..that was TurboC 1
0
 
LVL 45

Expert Comment

by:Kdo
Comment Utility

65536 is 0x10000. (0200000 to some of us.  :) )   17 bits.

It occurred to me last night that indexing the pointer by a value of 65536 or larger should cause an error.  Apparently not.

I'm really disappointed, too.  I taught myself C 21 years ago and never knew that huge pointers behaved this way.

sigh....

Kent
0
 
LVL 17

Expert Comment

by:rstaveley
Comment Utility
1977: 8086 previewed
1981: IBM's first 8086 PC with QDOS
1985: 80386 and Windows
1989: Internet
1991: Linux 0.01
2003: Kdo and rstaveley grapple with 8086 architecture :-)
0
 
LVL 4

Author Comment

by:meet_zorrer
Comment Utility
Hold your horses tight guys !
0
 
LVL 4

Author Comment

by:meet_zorrer
Comment Utility
Hold your horses tight guys !
0
 
LVL 4

Author Comment

by:meet_zorrer
Comment Utility
KDO ! if you have better solution you are welcomed too!
0
 
LVL 45

Expert Comment

by:Kdo
Comment Utility

My "better solution" would fit right into the middle of rstavely's timeline.  Put the application on a Control Data NOS/VE system where an int is 64 bits and each task has 2**60 bits of virtual address space.

But I guess that's not realistic, is it?

Machines today are so fast the the small overhead of huge pointers is a non-issue.  Using huge pointers instead of far pointers would be my approach if you want to keep the application backward compatible through the entire PC family.  Granted, you DO have to make some small code changes.  Or for $69 you can buy Borland C++ Builder 6.0 which includes a 32 bit C compiler and all of the current pointer definitions are just fine.  The downside here is that it doesn't support the functions prototyped in conio.h.


Decisions, decisions.....

Kent
0
 
LVL 17

Expert Comment

by:rstaveley
Comment Utility
> Control Data NOS/VE system

Was that one of Cray's creations? I don't suppose you ever got one of those manufactured as a laptop.
0
 
LVL 45

Expert Comment

by:Kdo
Comment Utility

Actually, Cray was gone long before this system was built.  But like the Cray series, it was a 64-bit system.

0
 
LVL 4

Author Comment

by:meet_zorrer
Comment Utility
>>>>>>>  Or for $69 you can buy Borland C++ Builder 6.0 <<<<<<<

Well Are you sure there will be no problems in running Borland C v 3.1 for DOS code in Borland C++ Builder 6.0 (as i am not using any special services of conion.h i can change that code) with out making big Structure changes in my Codings?

If YES then i donot have any problem buying Borland C++

Regards,
Meet
0
 
LVL 45

Expert Comment

by:Kdo
Comment Utility

I've got C++ Builder 5.x installed on several machines and should upgrade them to 6.0 in the next few weeks.  The only problems that I've had compiling C code that Borland's other compilers handle is when a function is missing in the C++ library and that has always been related to the console functions prototyped in conio.h.  gotoxy(), clrscr(), gettext(), etc.....

Besides, after using the builder for a while I've discovered that I actually enjoy Window's programming.  [shudder]

Kent
0
 
LVL 4

Author Comment

by:meet_zorrer
Comment Utility
So the answer is NO if we have to just YES or NO.

First of all the Borland C++ Builder is not good C code compiler and i sure have to edit my code, so i again go with  Rstaveley!
0
 
LVL 45

Expert Comment

by:Kdo
Comment Utility

Hi meet_zorrer,

I'm not sure how to interpret your last response.

C++ Builder includes a C compiler that is compatible with other Borland C compilers.  (More compatible than other vendor's compilers.)  You should be able to compile and link it more easily with a Borland product than any other, especially if the code is non-ANSI.

Even the use of obsolete entities like conio.h is usually easier when staying within a vendor's product line.


Kent
0
 
LVL 17

Expert Comment

by:rstaveley
Comment Utility
You can download BC 5.5 for free from http://www.borland.com/products/downloads/download_cbuilder.html if you don't want to shell out $69 for version 6.0. It is 8.7M. Nothing ventured, nothing gained.

The advantage of using this compiler is that you can port your DOS command line applications to Win32 console programs, which have access to 32-bit memory. 16-bit programming really is a thing of the past.

I use Microsoft Visual C++ for Windows C/C++ development myself, because someone else footed the bill for it.
0
 
LVL 4

Author Comment

by:meet_zorrer
Comment Utility
OK all you gyes out there ! it my decisian time now, so i am goin to use the binary/randome file for my program as techniques you guys have given about  working in memory are some way tough too.

Also guys KDO, Rstaveley both of your solutinos are up to the mark, it's only me, that is even living in 2003, using ancient Borland C v3.1  compiler

so thanks both of guys

so points are of Rstaveley, but as KDO was good too,
so i how can i give both of you points tell me?

or if moderator is viewing kindly close the question and give both of them points ;)

Regards,
Meet.
0
 
LVL 45

Expert Comment

by:Kdo
Comment Utility

Hi meet_zorrer,

You're not the only one living before 2003 with a Borland Turbo C compiler.  ;)

I've got three of them and obviously rstavely has at least two.

And for some "quick and dirty" C programs, it's faster to write them in Turbo C than with their newer compilers.  Of course, that's not always true.


Kent
0
 
LVL 17

Expert Comment

by:rstaveley
Comment Utility
To split points....

http:/help.jsp#hi69
0
 
LVL 4

Author Comment

by:meet_zorrer
Comment Utility
I was so confused about the solution of my problem that i donot download and tried new compilers, as mentioned Watcom C and LCC..

but
Comment from g0rath
Date: 11/24/2003 05:40AM PST
 Comment  

LCC-Win32 is also a free C compiler that runs native under windows

http://www.cs.virginia.edu/~lcc-win32/

LCC is amazing, Awosome, Coool and rocks, it just runs my code as it is without even changing the #include ... lines of my code and all is well in it except for just C help file, but no problem i code in Borland C IDE and then before final release i compile my code in LCC...... Simple as banana milk shake ):-D

So as soon as i get points, i will give them to [ g0rath ]

Thanks a lot g0rath!!
0
 
LVL 17

Expert Comment

by:rstaveley
Comment Utility
My only misgiving about using LCC-Win32 is that it has a relatively small user group. When things are going well, you can enjoy its features, but when you get snagged, you may miss out on support.

The GNU C compiler (gcc for C and g++ for C++)  has a *huge* user group, which is mostly on UN*X, but Win32 users are also numerous (Cygwin or MinGW).
0
 
LVL 4

Author Comment

by:meet_zorrer
Comment Utility
Hi rstaveley! i am an intermediate programmer in C, and i donot want any kind of support regarding LCC compiler as long as it can compile my C code. And when it doesn't compile i switch the compiler again (may be to GCC) as i have switched from Borland C to LCC.

It all depends on the time frame in which we (custom software)developers work. What i am always conserned and worried about is my coding not the support, as i have to support my self and give it to my application users. So even that was why i didn't even download the LCC and Watcome compilers in the course of my problem.

When i get my time back i give a look at LCC and it was just one fine small good looking Compiler and Editor.

And it is hard for me to setup Cygwin and blah blah on my pc just to get my code compiled down to more beautifull Win32 Executable.

Regards,
Meet
0
 
LVL 17

Expert Comment

by:rstaveley
Comment Utility
Hope it all goes well for you Meet :-)
0
 
LVL 45

Expert Comment

by:Kdo
Comment Utility

Hi meet_zorrer,

Changing development environments is an awful burden on a staff.  You wind up with come code that works with one compiler and other code that works with another.  Carry this out a bit and you get an application that requires several different C compilers to build.  This makes maintenance of the application difficult and ultimately puts you in a situation where the senior developers can hold management hostage by developing their own processes and procedures for maintaining the application.  The junior developers aren't in on the secret and the senior one's don't share so they continue to look like gurus while the new-hires continue to look incompetent.

One of the potential problems with changing compilers is that compilers are just programs and subject to their own bugs.  It's not uncommon to have code that works just fine when built with one compiler fall flat when built with another.

If it were my application and I intended for it to have commercial value, I'd certainly insist that the development environment had some measure of support.  Either stick with Borland or Micro$oft, or use gcc, the most widely used GPL compiler with a huge support mechanism.


Good Luck,
Kent
0

Featured Post

Highfive + Dolby Voice = No More Audio Complaints!

Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

Join & Write a Comment

This tutorial is posted by Aaron Wojnowski, administrator at SDKExpert.net.  To view more iPhone tutorials, visit www.sdkexpert.net. This is a very simple tutorial on finding the user's current location easily. In this tutorial, you will learn ho…
Windows programmers of the C/C++ variety, how many of you realise that since Window 9x Microsoft has been lying to you about what constitutes Unicode (http://en.wikipedia.org/wiki/Unicode)? They will have you believe that Unicode requires you to use…
The goal of this video is to provide viewers with basic examples to understand and use structures in the C programming language.
The goal of this video is to provide viewers with basic examples to understand how to use strings and some functions related to them in the C programming language.

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now