Memcpy > 64K

Posted on 1999-01-25
Last Modified: 2008-02-01
In a 16-bit Windows app, I have allocated a gloabl memory block of 90000 bytes (array of 300 elements of 300 bytes each).
The memory is allocated OK, but when I try to memcpy something into element 218, the application crashes with a GP.
Probably something to do with the 64K limit. How can I work my way around this.
I use the Boralnd C++ 4.52 compiler, and compile in large memory model.
Question by:parlando
  • 5
  • 2
  • 2
  • +4
LVL 10

Expert Comment

ID: 1258345
2 tries since I don't know this compiler.
1.) Use the huge memory modell
2.) Many Dos-C compiler have a fmemcpy (f stands for far). Check your menuals (help-files) for this function.

Author Comment

ID: 1258346
1) Using the huge memory model involves too many changes to my application(s).
2) _fmemcpy didn't help.
I've worked my way around it by chopping the block into smaller pieces, wich was a possibility in this case.
The urgency for a solution has gone, but I'm still interested in solutions for future use.

Expert Comment

ID: 1258347
Try creating two huge pointers, then
1) copy the first 64kB
2) add 64kB to both pointers
3) copy the rest

I don't know if it works, the problem is, that you need to change the descriptors (1st 16-bit of pointers), if the add does that, it should be ok.

Author Comment

ID: 1258348
It's not that I need to copy large blocks of data, because the allocation of memory was succesful (it's a Windows application).
It's just that I can't copya block of 300 bytes (a record) past the 64K limit.

Expert Comment

ID: 1258349
It is still the same problem.
In 16 bit, a far pointer has 2 parts.
The 1st 16 bits selects a (max.) 64kB block of memory. It is called a segment in real mode, and a selector in protected mode.
The 2nd 16 bits are an offset.

When you need to access data beyond 64kB, you can't just add something to your pointer, because you'll either get a wrong offset (if you only add to the 2nd 16 bits), or the 1st 16 bits of your pointer will become invalid.

In BC++ 3.1 real mode, you used huge pointers to solve this problem. (In real mode, you just need to do your calculation a bit differently. When you declare a pointer as huge, the compiler uses this method, which you only use when needed, because it is a lot slower).

In protected mode (which I think you use), you can't tell what value you should put into the 1st 16 bits, because it depends on which os you use. If BC++ knows how, the huge pointer approch should work. If not, there must be some Windows function, which can do it for you.

Expert Comment

ID: 1258350
Assuming your 300-byte array item is a struct called myStruct:

//Allocate array of 300 pointers.
myStruct *myStructArray = malloc(300 * sizeof (myStruct *)) ;

//use farmalloc to allocate each individual entry
int x;
for (x=0; x < 300; x++)
   myStructArray[x] = farmalloc(sizeof(myStruct));

Free 300-byte structures and list before quitting program.
for (x=0; x < 300; x++)

Note: be careful when accessing your structures (that now live on the far heap outsize of the 64K data segment). Things like strcpy will not work as need to use the far versions.

Expert Comment

ID: 1258351
Since you specifically said memcpy, the far version (if you decide to use my suggested approach) is _fmemcpy().
Zoho SalesIQ

Hassle-free live chat software re-imagined for business growth. 2 users, always free.


Author Comment

ID: 1258352
It's just things like strcpy (or memcpy in my case) that cause the problems. The memory allocation itself works fine.

Expert Comment

ID: 1258353
The size parameter of the memcpy() function has type size_t. I bet a size_t on
your machine is defined as a 16 bit unsigned int. BTW, the parameter for malloc()
happens to be a size_t too; are you definitely, positively, absolutely, totally
sure that your malloc succeeded? IOW can your malloc allocate more than 64K bytes?

kind regards,

Jos aka

Author Comment

ID: 1258354
Absolutely positively sure! It's a Windows application, and I use GlobalAlloc().

Accepted Solution

SaGS earned 100 total points
ID: 1258355
To simplify the discussion, we assume you want to obtain something like this:

struct s { char a[288]; int i; } ary [300];
strcpy (ary[218].a, "Some long string .......");
ary[219] = ary[218];

The problem is this element crosses a 64k boundary. No (or almost no) C run-time function accepts this except in 'huge' mem model.

First of all, the array itself must be declared with the __huge modifier, otherwise &ary[219] will not be computed correctly. For the same reason, any pointer to this array must be __huge.

struct s { char a[288]; int i; } __huge ary [300];

After taking care of this, you can remain into any memory model and use one of the following workarounds:

(1) Make the array element to be a power of two. This way no element crosses a 64k boundary, and each individual element of the array can be treated as __far data. Notice this is the only way you can be sure that references to structure members ("p = &ary[218]; p->i = 100;" and even "ary[218].i = 100;") will work. Becasue of the overhead needed to take into account crossing a 64k barrier, compilers suppose that a single object (simple variable or structure) does never cross such a barrier; this is true even for the 'huge' memory model.

struct s { char a[288]; int i; char gap[212]; } __huge ary [300];
_fstrcpy (ary[218].a, "Some long string .......");
ary[219] = ary[218];

(2) Use the hmemcpy() Windows API to read/write data from the structure, which allows crossing segments (and byte count up to 2Gig-1). These work (notice that for strings the count passed must include the terminating '\0'):

hmemcpy (ary[218].a, "Some long string .......", strlen "Same long string ......." +1);
hmemcpy (&ary[219], &ary[218], sizeof (struct s));

but these don't:

ary[219] = ary[218];
ary[218].i = 100;


Expert Comment

ID: 1258356
Difficult to tell what's wrong without seeing the code, but you'll definitely need huge pointers to do what you intended first.

See this :

h = GlobalAlloc (200000,..);
char huge *hp;
char far *fp;
hp = GlobalLock (h);
hp += 64*1024L; // offsets hp by 64K => succeeds (1)
fp += 64*1024L; // offsets fp by 64K => fails (2)

In (1), the compiler generates all the code necessary to move a huge pointer across the segment boundary. Line (2) will because the compiler will treat fp as a 'normal' pointer and will simply increment its value.

Also, if I remember well, there is no huge version of the std C lib in Borland. Hence, all C library calls (strcpy, etc.) will see a huge pointer as a far, yielding unexpected (and generally disastrous) results.

Expert Comment

ID: 1258357
wpd is right about the 'huge' memory model libraries, and my statement "No (or almost no) C run-time function accepts this except in 'huge' mem model" is not exact.

With 'huge' model, the 'large' model libraries are used. However, the docs that came with my compiler (this info is implementation-specific) state that the following functions within the large model library support huge pointers: bsearch, fread, fwrite, _fmemccpy, _fmemchr, _fmemcmp, _fmemcpy, _fmemicmp, _fmemmove, _fmemset, _halloc, _hfree, _lfind, _lsearch, _memccpy, memchr, _memicmp, memmove, qsort, memcmp, memcpy, memset. The last 3 of these also have intrinsic versions (the compiler generates inline code instead of calling a function), but the inline versions do not support huge pointers.

(My HTML formatting test. Disregarg the following text.
<B>bold</B>, <I>italic</I>,<PRE>
    preformatted text,    non roportional font
link to <A HREF="">this page</A>.)

Author Comment

ID: 1258358
Thanks guys,
With this information I'll be able to solve the problem.I've learned a few new things about (huge) memory models and restrictions.

Featured Post

Best Practices: Disaster Recovery Testing

Besides backup, any IT division should have a disaster recovery plan. You will find a few tips below relating to the development of such a plan and to what issues one should pay special attention in the course of backup planning.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
reading tzdatabase for timezone definitions 5 128
Way to improve it 16 61
Need example 5 107
delete-remove 14 60
Windows programmers of the C/C++ variety, how many of you realise that since Window 9x Microsoft has been lying to you about what constitutes Unicode ( They will have you believe that Unicode requires you to use…
This is a short and sweet, but (hopefully) to the point article. There seems to be some fundamental misunderstanding about the function prototype for the "main" function in C and C++, more specifically what type this function should return. I see so…
Video by: Grant
The goal of this video is to provide viewers with basic examples to understand and use nested-loops in the C programming language.
Video by: Grant
The goal of this video is to provide viewers with basic examples to understand and use while-loops in the C programming language.

932 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

21 Experts available now in Live!

Get 1:1 Help Now