• C

MALLOC > 64K

Hi to All!

I want to allocate memory in pure C code (no OOP) as maximum as possible i have 128MB ram in pc but i can only occupy 64k max,

I am using Borland C v 3.1 Compiler under windows 98,

and if it is not possible in Borland C then there will be some other compiler out there, just give a link to download or any thing and code too,
                                                  BUT
again C (pure C) compilers what i want, as i donot want to change my exiting C code.
LVL 4
meet_zorrerAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

sunnycoderCommented:
Hi meet_zorrer,

I guess any windows/linux based compiler should do it for you ...

you can either use VC++ or gcc .... VC++ is ofcourse M$ product and gcc port can be downloaded from gcc.gnu.org

Cheers!
Sunny:o)
0
meet_zorrerAuthor Commented:
Is GCC Pure C compiler or OOP only or both?

and what about the code???

PS: i donot want VC++, i already have in my PC
0
sunnycoderCommented:
gcc compiles C as well as C++ ... I think front ends for fortran and pascal are also being developed (or is it that they have been developed ) ...

to ensure that it compiles pure ANSI C, compile your .c files with -pedantic flag
0
Introducing the "443 Security Simplified" Podcast

This new podcast puts you inside the minds of leading white-hat hackers and security researchers. Hosts Marc Laliberte and Corey Nachreiner turn complex security concepts into easily understood and actionable insights on the latest cyber security headlines and trends.

meet_zorrerAuthor Commented:
there are lot of versions of gcc what sould i use?? the latest one or some other stable version?
0
sunnycoderCommented:
the most recent stable version will be a good choice ... i am using 2.96 and works fine for me (note I am on Linux)
0
meet_zorrerAuthor Commented:
i read here and there on gcc site but there is no binaries available for the windows but the cygwin project from http://sources.redhat.com/cygwin/ so i thing we are goin off track now to the question , as the gcc is not for windows }:-(
0
sunnycoderCommented:
gcc was originally built for linux but it has been ported to a number of platforms ... cygwin is linux simulator for windows ...
I have used VC++ for sometime in past but I like gcc better ... I am not sure if installing gcc on win requires you to have cygwin too ...

Wait for some windows guru to comment on this
0
grg99Commented:
If you need to use more than 64K, you'll need a compiler that can generate 32-bit protected-mode code.

I'd suggest Watcom C, as it's open source, free, and can do almost any memory model you might need.

www.openwatcom.org  I think is the place.

0
g0rathCommented:
LCC-Win32 is also a free C compiler that runs native under windows

http://www.cs.virginia.edu/~lcc-win32/
0
Kent OlsenData Warehouse Architect / DBACommented:

Hi meet_zorrer,

Borland's Turbo C (which is what you are using) is a 16 bit compiler.  This means that it has 16-bit addressing.  The maximum value that you can keep in 16 bits is 65535, hence you have a 64K limitation.

Granted, you said that you don't want to change your C code, but to get more than 64K you're going to have to move off of the 16-bit platform (using one of the 32-bit C compilers that has been mentioned) or make a small change to the code.

Borland's memory management routines have a faralloc() function that allows you to reserved more than 64K in a single block.  The function is a direct replacement for malloc(), except that the argument is a long int so values greater than 64K can be accomodated.

But be cautioned -- memory addressing is still a 16-bit operation.  If you faralloc() a buffer of (for example) 80K you'll need to play some tricks to get to the memory addresses above 64K.  The following code doesn't do what you want:

char *Buffer, *Ptr;

Buffer = faralloc(80000);
Ptr = Buffer + 70000;

Because memory is 16-bit, only the lower 16 bits of the value 70000 will be added to the pointer.  Ptr will actually point to Buffer + 4464.  (4464 is modulo (70000, 65536).)

But you can "work around" this quite effectively.

Buffer = faralloc (80000);
Ptr = Buffer + 32768;       /*  32768 is 65536/2  */
Ptr = Buffer + 32768;

Ptr now points to the first byte in the region above 65536.


Good Luck,
Kent
0
rstaveleyCommented:
farmalloc returns far pointers, which can theoretically address any part of your 1M DOS (= 24 bit) memory address space. You get the same pointers returned from malloc if you are using a medium or large memory model, but you do not need the far keyword, because data pointers are implicitly far.

These 32-bit pointers are cranky, as indicated by Kent, because they have 16-bits for the segment and 16-bits for the offset. Far pointer arithmetic leaves the segment unchanged and only applies changes the offset, because who in their right mind would need more than 64K ;-), and it is generally optimal to work that way. Mostly far pointers are used to point to < 64K chunks of memory, but by having far pointers you don't have to have the sum of those chunks totalling < 64K.

If you want to address a chunk of memory which is > 64K, you should use a huge pointer. Huge pointers automatically do normalisation for you when you do pointer arithmetic... or rather the compiler adds code to normalise huge pointers. "Nomalising" means that you put as much of the 24 bits address into the segment as possible and have only the least significant nibble of the offset left with significance.

e.g. 1000:1234 is normalised to 1123:0004, which points to the same address, but leaves more room in the offset to do arithmetic.

I believe that you can directly index element 65536 of a char array pointed to by a huge pointer, but... it seems like a long time since I did work with DOS.

        char huge *chptr = (char huge*)farmalloc(80000);
        char far* cfptr = &chptr[65536]; /* I think that this works OK... but you should check :-) */
0
Kent OlsenData Warehouse Architect / DBACommented:

Hi rstavely,

faralloc(), farmalloc() -- not sure if that was a slip of the fingers of a slip of the mind.  :)  Thanks for the correction.

Having huge pointers does make for certain niceties, but you still cant "directly" address more than 64K.  16 bits is 16 bits and address calculations are performed with 16 bit math.  (At least I don't recall the generated code jumping through hoops to do 24/32-bit math for huge pointers.)


Kent
0
grg99Commented:
The farmalloc() functions are fine, but they only get you access to DOS memory, which usually tops out around 600K.

Well, actually, I lie, there are ways to access more memory, but they're not of general use, they require a lot of calls to copy blocks
back and forth, and it's not only slow, it's a pain to use.

To get access to ALL your memory, you will need to make the jump to protected mode, either 16 or 32 bit varieties.

0
rstaveleyCommented:
I've just dug up an old Borland compiler (Turbo C version 1 !!), and I can see that it does indeed do kosher pointer arithmetic in the example I gave.

Here's the code snippet:
--------8<--------
      char huge *chptr = (char huge*)farmalloc(80000);
      char far* cfptr = &chptr[65536]; /* I think that this works OK... but you should check :-) */
      *cfptr = 1;
--------8<--------

Here's the generated assembler code:
--------8<--------
      mov      dx,1
      mov      ax,14464
      push      dx
      push      ax
      call      near ptr _farmalloc
      pop      cx
      pop      cx
      mov      word ptr [bp-6],dx
      mov      word ptr [bp-8],ax
; Line 6
      mov      dx,word ptr [bp-6]
      mov      ax,word ptr [bp-8]
      mov      cx,1                               ; <- Here's it putting 0x10000 into cx:bx
      xor      bx,bx
      call      far ptr padd@                ; <- ..to put the correct address into dx:ax
      mov      word ptr [bp-2],dx
      mov      word ptr [bp-4],ax
      les      bx,dword ptr [bp-4]
      mov      byte ptr es:[bx],1           ; <- Here's the assignment
--------8<--------
The pointer arithmetic is done by an internal function padd@.

It does jump through some hoops, but isn't it great *not* to have to work with 24 bit code nowadays? :-)

Incidentally, none of this will address the 128MB, meet_zorrer. Virtual 86 DOS boxes in Windows '98 will only give you 640K to play with. You really ought to get a copy of GCC as previously mentioned - see http://www.mingw.org/
0
Kent OlsenData Warehouse Architect / DBACommented:

Cool.  You learn something new (or old... very very old....) every day.  :)

BTW, I too, have distros for Turbo C 1, 2, and 3.

Kent
0
rstaveleyCommented:
Remember how Turbo C 1 could fit on a bootable 720K floppy with space for PKZip, QEdit and an obligatory boot sector virus? There was no need for a lap-top when visiting installations with that in your shirt pocket! Makes me go all misty-eyed ;-)
0
Kent OlsenData Warehouse Architect / DBACommented:
All of my Turbo C distributions came on 5 inch floppies.  (Am I the only bithead in the world that calls them "5 inch" floppies instead of "5 1/4 inch"?  If anyone ever markets a REAL 5-inch floppy I'll change with the times.)  IIRC, Turbo C 3.x is on 3 floppies plus Turbo Debugger on 2 more and Turbo Assembler on another.

Quite a difference between those and C++ Builder's 100MB+ install file.  :)

0
meet_zorrerAuthor Commented:
All guys out there Thanks,
                           BUT

my problem is still there and even becoming less it gets up as now i have to setup the GCC and CYGWIN/ or learn new compilers like WahtCom etc...

what is the best round about if i have to stick to Borland C only , i am using Double link lists and memory goes off after about 10 to 15 nodes and the size of node is about 2 KB

so what sould be my new Plan:

1) Use the Random/Binary File handling forget about Memory
2) Use new compiler
3) or Donot use the Datastructure. use the Borland C as a 1 month old programer does it


ok Guys i am starting to get worried now! :-?
0
rstaveleyCommented:
If ~200KB is enough for you (allowing for a ~300K program size), use farmalloc. If it isn't, you're going to have to bite the bullet.
0
sunnycoderCommented:
Unless your job or development environment revolves around 16 bit compilers, I would suggest move on to 32 bit compilers as they are the ones in most widespread use (like it or not)
0
meet_zorrerAuthor Commented:
well i donot like it  }:-?
0
rstaveleyCommented:
If you are loading a relatively large array of relatively small structs into RAM, you might consider reading and writing to a database.

e.g.

    static struct YOURSTRUCT current_record;
    int current_record_recnum; /* Record number of current record */
    int current_record_modified; /* Non-zero if the current record has been modified */
    FILE *file; /* Read/write binary file - the database - i.e. file = fopen("database.dat",r+b); */

    int set_current_record(int recnum)
    {
        /* Write current record to file, if it has been modified */
        if (current_record_modified) {
                if (fseek(file,sizeof(struct YOURSTRUCT)*current_recnum,SEEK_SET) != 0)
                        return 0;
                if (fwrite(&current_record,sizeof(struct YOURSTRUCT),1,file) == 1)
                        current_record_modified = 0;
        }

        /* Read record recnum from the file into the current record */
        if (current_record_recnum != recnum) {
                if (fseek(file,sizeof(struct YOURSTRUCT)*recnum,SEEK_SET) != 0)
                        return 0;
                if (fread(&current_record,sizeof(struct YOURSTRUCT),1,file) == 1)
                        current_record_modified = 0;
                else
                        return 0;
        }
        return 1;
    }

Do remember to use a medium or large memory model to ensure that your total data isn't confined to a 64K data segment.
0
meet_zorrerAuthor Commented:
ok rstaveley! if your solution works then i will be a happy man again, I try it tonight so points gona be yours as i just want to get round about my problem!
0
Kent OlsenData Warehouse Architect / DBACommented:

Hey -- rstavely....


> I've just dug up an old Borland compiler (Turbo C version 1 !!), and I can see that it does indeed do kosher pointer arithmetic in the example I gave.

> Here's the code snippet:
> --------8<--------
>      char huge *chptr = (char huge*)farmalloc(80000);
>      char far* cfptr = &chptr[65536]; /* I think that this works OK... but you should check :-) */
>      *cfptr = 1;
> --------8<--------

Do I smell a fast one being pulled here?  :)  TurboC should throw a compilation error when you try to index an array by a 17 bit value ????

char far* cfptr = chptr +65535;  /*  This ??might?? work  */


Kent

0
rstaveleyCommented:
17 bit??

The generated assembler code was indeed generated. You can see CX:BX being used to pass the long index.

Indexing the huge pointer works with a long, but indexing the far pointer doesn't:
--------8<--------
#include <stdio.h>
#include <alloc.h>

main()
{
      char huge *chptr = (char huge*)farmalloc(80000);
      char far* cfptr = (char far*)&chptr[65537];
      char far* cfptr2 = (char far*)&chptr[0];
      char far* cfptr3 = &cfptr2[65537];

      *cfptr = 123;
      printf("It is as easy as %d when you index the huge pointer\n",(int)chptr[65537]);
      printf("Huge pointer is %Fp\n",chptr);
      printf("Far pointer pointing to element 0x10001 is %Fp\n\n",cfptr);

      *cfptr3 = 0;
      printf("There are %d problems when you index the far pointer\n",(int)chptr[65537]);
      printf("Huge pointer is %Fp\n",chptr);
      printf("Far pointer used for indexing is %Fp\n",cfptr2);
      printf("Far pointer pointing to element 0x10001 is %Fp\n\n",cfptr3);
}
--------8<--------

Gets...
--------8<--------
It is as easy as 123 when you index the huge pointer
Huge pointer is 193E:0004
Far pointer pointing to element 0x10001 is 293E:0005

There are 123 problems when you index the far pointer
Huge pointer is 193E:0004
Far pointer used for indexing is 193E:0004
Far pointer pointing to element 0x10001 is 193E:0005
--------8<--------

The huge pointer indexed by a long returns a normalised pointer. The far pointer indexed by a long... is only indexed by the least significant 16-bits of the long.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
rstaveleyCommented:
..that was TurboC 1
0
Kent OlsenData Warehouse Architect / DBACommented:

65536 is 0x10000. (0200000 to some of us.  :) )   17 bits.

It occurred to me last night that indexing the pointer by a value of 65536 or larger should cause an error.  Apparently not.

I'm really disappointed, too.  I taught myself C 21 years ago and never knew that huge pointers behaved this way.

sigh....

Kent
0
rstaveleyCommented:
1977: 8086 previewed
1981: IBM's first 8086 PC with QDOS
1985: 80386 and Windows
1989: Internet
1991: Linux 0.01
2003: Kdo and rstaveley grapple with 8086 architecture :-)
0
meet_zorrerAuthor Commented:
Hold your horses tight guys !
0
meet_zorrerAuthor Commented:
Hold your horses tight guys !
0
meet_zorrerAuthor Commented:
KDO ! if you have better solution you are welcomed too!
0
Kent OlsenData Warehouse Architect / DBACommented:

My "better solution" would fit right into the middle of rstavely's timeline.  Put the application on a Control Data NOS/VE system where an int is 64 bits and each task has 2**60 bits of virtual address space.

But I guess that's not realistic, is it?

Machines today are so fast the the small overhead of huge pointers is a non-issue.  Using huge pointers instead of far pointers would be my approach if you want to keep the application backward compatible through the entire PC family.  Granted, you DO have to make some small code changes.  Or for $69 you can buy Borland C++ Builder 6.0 which includes a 32 bit C compiler and all of the current pointer definitions are just fine.  The downside here is that it doesn't support the functions prototyped in conio.h.


Decisions, decisions.....

Kent
0
rstaveleyCommented:
> Control Data NOS/VE system

Was that one of Cray's creations? I don't suppose you ever got one of those manufactured as a laptop.
0
Kent OlsenData Warehouse Architect / DBACommented:

Actually, Cray was gone long before this system was built.  But like the Cray series, it was a 64-bit system.

0
meet_zorrerAuthor Commented:
>>>>>>>  Or for $69 you can buy Borland C++ Builder 6.0 <<<<<<<

Well Are you sure there will be no problems in running Borland C v 3.1 for DOS code in Borland C++ Builder 6.0 (as i am not using any special services of conion.h i can change that code) with out making big Structure changes in my Codings?

If YES then i donot have any problem buying Borland C++

Regards,
Meet
0
Kent OlsenData Warehouse Architect / DBACommented:

I've got C++ Builder 5.x installed on several machines and should upgrade them to 6.0 in the next few weeks.  The only problems that I've had compiling C code that Borland's other compilers handle is when a function is missing in the C++ library and that has always been related to the console functions prototyped in conio.h.  gotoxy(), clrscr(), gettext(), etc.....

Besides, after using the builder for a while I've discovered that I actually enjoy Window's programming.  [shudder]

Kent
0
meet_zorrerAuthor Commented:
So the answer is NO if we have to just YES or NO.

First of all the Borland C++ Builder is not good C code compiler and i sure have to edit my code, so i again go with  Rstaveley!
0
Kent OlsenData Warehouse Architect / DBACommented:

Hi meet_zorrer,

I'm not sure how to interpret your last response.

C++ Builder includes a C compiler that is compatible with other Borland C compilers.  (More compatible than other vendor's compilers.)  You should be able to compile and link it more easily with a Borland product than any other, especially if the code is non-ANSI.

Even the use of obsolete entities like conio.h is usually easier when staying within a vendor's product line.


Kent
0
rstaveleyCommented:
You can download BC 5.5 for free from http://www.borland.com/products/downloads/download_cbuilder.html if you don't want to shell out $69 for version 6.0. It is 8.7M. Nothing ventured, nothing gained.

The advantage of using this compiler is that you can port your DOS command line applications to Win32 console programs, which have access to 32-bit memory. 16-bit programming really is a thing of the past.

I use Microsoft Visual C++ for Windows C/C++ development myself, because someone else footed the bill for it.
0
meet_zorrerAuthor Commented:
OK all you gyes out there ! it my decisian time now, so i am goin to use the binary/randome file for my program as techniques you guys have given about  working in memory are some way tough too.

Also guys KDO, Rstaveley both of your solutinos are up to the mark, it's only me, that is even living in 2003, using ancient Borland C v3.1  compiler

so thanks both of guys

so points are of Rstaveley, but as KDO was good too,
so i how can i give both of you points tell me?

or if moderator is viewing kindly close the question and give both of them points ;)

Regards,
Meet.
0
Kent OlsenData Warehouse Architect / DBACommented:

Hi meet_zorrer,

You're not the only one living before 2003 with a Borland Turbo C compiler.  ;)

I've got three of them and obviously rstavely has at least two.

And for some "quick and dirty" C programs, it's faster to write them in Turbo C than with their newer compilers.  Of course, that's not always true.


Kent
0
rstaveleyCommented:
To split points....

http:/help.jsp#hi69
0
meet_zorrerAuthor Commented:
I was so confused about the solution of my problem that i donot download and tried new compilers, as mentioned Watcom C and LCC..

but
Comment from g0rath
Date: 11/24/2003 05:40AM PST
 Comment  

LCC-Win32 is also a free C compiler that runs native under windows

http://www.cs.virginia.edu/~lcc-win32/ 

LCC is amazing, Awosome, Coool and rocks, it just runs my code as it is without even changing the #include ... lines of my code and all is well in it except for just C help file, but no problem i code in Borland C IDE and then before final release i compile my code in LCC...... Simple as banana milk shake ):-D

So as soon as i get points, i will give them to [ g0rath ]

Thanks a lot g0rath!!
0
rstaveleyCommented:
My only misgiving about using LCC-Win32 is that it has a relatively small user group. When things are going well, you can enjoy its features, but when you get snagged, you may miss out on support.

The GNU C compiler (gcc for C and g++ for C++)  has a *huge* user group, which is mostly on UN*X, but Win32 users are also numerous (Cygwin or MinGW).
0
meet_zorrerAuthor Commented:
Hi rstaveley! i am an intermediate programmer in C, and i donot want any kind of support regarding LCC compiler as long as it can compile my C code. And when it doesn't compile i switch the compiler again (may be to GCC) as i have switched from Borland C to LCC.

It all depends on the time frame in which we (custom software)developers work. What i am always conserned and worried about is my coding not the support, as i have to support my self and give it to my application users. So even that was why i didn't even download the LCC and Watcome compilers in the course of my problem.

When i get my time back i give a look at LCC and it was just one fine small good looking Compiler and Editor.

And it is hard for me to setup Cygwin and blah blah on my pc just to get my code compiled down to more beautifull Win32 Executable.

Regards,
Meet
0
rstaveleyCommented:
Hope it all goes well for you Meet :-)
0
Kent OlsenData Warehouse Architect / DBACommented:

Hi meet_zorrer,

Changing development environments is an awful burden on a staff.  You wind up with come code that works with one compiler and other code that works with another.  Carry this out a bit and you get an application that requires several different C compilers to build.  This makes maintenance of the application difficult and ultimately puts you in a situation where the senior developers can hold management hostage by developing their own processes and procedures for maintaining the application.  The junior developers aren't in on the secret and the senior one's don't share so they continue to look like gurus while the new-hires continue to look incompetent.

One of the potential problems with changing compilers is that compilers are just programs and subject to their own bugs.  It's not uncommon to have code that works just fine when built with one compiler fall flat when built with another.

If it were my application and I intended for it to have commercial value, I'd certainly insist that the development environment had some measure of support.  Either stick with Borland or Micro$oft, or use gcc, the most widely used GPL compiler with a huge support mechanism.


Good Luck,
Kent
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
C

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.