Solved

Memcpy > 64K

Posted on 1999-01-25
14
876 Views
Last Modified: 2008-02-01
In a 16-bit Windows app, I have allocated a gloabl memory block of 90000 bytes (array of 300 elements of 300 bytes each).
The memory is allocated OK, but when I try to memcpy something into element 218, the application crashes with a GP.
Probably something to do with the 64K limit. How can I work my way around this.
I use the Boralnd C++ 4.52 compiler, and compile in large memory model.
0
Comment
Question by:parlando
  • 5
  • 2
  • 2
  • +4
14 Comments
 
LVL 10

Expert Comment

by:rbr
ID: 1258345
2 tries since I don't know this compiler.
1.) Use the huge memory modell
2.) Many Dos-C compiler have a fmemcpy (f stands for far). Check your menuals (help-files) for this function.
0
 

Author Comment

by:parlando
ID: 1258346
1) Using the huge memory model involves too many changes to my application(s).
2) _fmemcpy didn't help.
I've worked my way around it by chopping the block into smaller pieces, wich was a possibility in this case.
The urgency for a solution has gone, but I'm still interested in solutions for future use.
0
 
LVL 1

Expert Comment

by:hustch
ID: 1258347
Try creating two huge pointers, then
1) copy the first 64kB
2) add 64kB to both pointers
3) copy the rest

I don't know if it works, the problem is, that you need to change the descriptors (1st 16-bit of pointers), if the add does that, it should be ok.
0
 

Author Comment

by:parlando
ID: 1258348
It's not that I need to copy large blocks of data, because the allocation of memory was succesful (it's a Windows application).
It's just that I can't copya block of 300 bytes (a record) past the 64K limit.
0
 
LVL 1

Expert Comment

by:hustch
ID: 1258349
It is still the same problem.
In 16 bit, a far pointer has 2 parts.
The 1st 16 bits selects a (max.) 64kB block of memory. It is called a segment in real mode, and a selector in protected mode.
The 2nd 16 bits are an offset.

When you need to access data beyond 64kB, you can't just add something to your pointer, because you'll either get a wrong offset (if you only add to the 2nd 16 bits), or the 1st 16 bits of your pointer will become invalid.

In BC++ 3.1 real mode, you used huge pointers to solve this problem. (In real mode, you just need to do your calculation a bit differently. When you declare a pointer as huge, the compiler uses this method, which you only use when needed, because it is a lot slower).

In protected mode (which I think you use), you can't tell what value you should put into the 1st 16 bits, because it depends on which os you use. If BC++ knows how, the huge pointer approch should work. If not, there must be some Windows function, which can do it for you.
0
 
LVL 1

Expert Comment

by:jim_pettinato
ID: 1258350
Assuming your 300-byte array item is a struct called myStruct:

//Allocate array of 300 pointers.
myStruct *myStructArray = malloc(300 * sizeof (myStruct *)) ;

//use farmalloc to allocate each individual entry
int x;
for (x=0; x < 300; x++)
   myStructArray[x] = farmalloc(sizeof(myStruct));





Free 300-byte structures and list before quitting program.
for (x=0; x < 300; x++)
  farfree(myStructArray[x]);
free(myStructArray);


Note: be careful when accessing your structures (that now live on the far heap outsize of the 64K data segment). Things like strcpy will not work as expected..you need to use the far versions.
0
 
LVL 1

Expert Comment

by:jim_pettinato
ID: 1258351
Since you specifically said memcpy, the far version (if you decide to use my suggested approach) is _fmemcpy().
0
What Should I Do With This Threat Intelligence?

Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

 

Author Comment

by:parlando
ID: 1258352
It's just things like strcpy (or memcpy in my case) that cause the problems. The memory allocation itself works fine.
0
 
LVL 4

Expert Comment

by:jos010697
ID: 1258353
The size parameter of the memcpy() function has type size_t. I bet a size_t on
your machine is defined as a 16 bit unsigned int. BTW, the parameter for malloc()
happens to be a size_t too; are you definitely, positively, absolutely, totally
sure that your malloc succeeded? IOW can your malloc allocate more than 64K bytes?

kind regards,

Jos aka jos@and.nl
0
 

Author Comment

by:parlando
ID: 1258354
Absolutely positively sure! It's a Windows application, and I use GlobalAlloc().
0
 
LVL 4

Accepted Solution

by:
SaGS earned 100 total points
ID: 1258355
To simplify the discussion, we assume you want to obtain something like this:

struct s { char a[288]; int i; } ary [300];
strcpy (ary[218].a, "Some long string .......");
ary[219] = ary[218];

The problem is this element crosses a 64k boundary. No (or almost no) C run-time function accepts this except in 'huge' mem model.

First of all, the array itself must be declared with the __huge modifier, otherwise &ary[219] will not be computed correctly. For the same reason, any pointer to this array must be __huge.

struct s { char a[288]; int i; } __huge ary [300];

After taking care of this, you can remain into any memory model and use one of the following workarounds:

(1) Make the array element to be a power of two. This way no element crosses a 64k boundary, and each individual element of the array can be treated as __far data. Notice this is the only way you can be sure that references to structure members ("p = &ary[218]; p->i = 100;" and even "ary[218].i = 100;") will work. Becasue of the overhead needed to take into account crossing a 64k barrier, compilers suppose that a single object (simple variable or structure) does never cross such a barrier; this is true even for the 'huge' memory model.

struct s { char a[288]; int i; char gap[212]; } __huge ary [300];
_fstrcpy (ary[218].a, "Some long string .......");
ary[219] = ary[218];

or
(2) Use the hmemcpy() Windows API to read/write data from the structure, which allows crossing segments (and byte count up to 2Gig-1). These work (notice that for strings the count passed must include the terminating '\0'):

hmemcpy (ary[218].a, "Some long string .......", strlen "Same long string ......." +1);
hmemcpy (&ary[219], &ary[218], sizeof (struct s));

but these don't:

ary[219] = ary[218];
ary[218].i = 100;

0
 
LVL 2

Expert Comment

by:wpd
ID: 1258356
Difficult to tell what's wrong without seeing the code, but you'll definitely need huge pointers to do what you intended first.

See this :

h = GlobalAlloc (200000,..);
char huge *hp;
char far *fp;
hp = GlobalLock (h);
hp += 64*1024L; // offsets hp by 64K => succeeds (1)
fp += 64*1024L; // offsets fp by 64K => fails (2)

In (1), the compiler generates all the code necessary to move a huge pointer across the segment boundary. Line (2) will because the compiler will treat fp as a 'normal' pointer and will simply increment its value.

Also, if I remember well, there is no huge version of the std C lib in Borland. Hence, all C library calls (strcpy, etc.) will see a huge pointer as a far, yielding unexpected (and generally disastrous) results.
0
 
LVL 4

Expert Comment

by:SaGS
ID: 1258357
wpd is right about the 'huge' memory model libraries, and my statement "No (or almost no) C run-time function accepts this except in 'huge' mem model" is not exact.

With 'huge' model, the 'large' model libraries are used. However, the docs that came with my compiler (this info is implementation-specific) state that the following functions within the large model library support huge pointers: bsearch, fread, fwrite, _fmemccpy, _fmemchr, _fmemcmp, _fmemcpy, _fmemicmp, _fmemmove, _fmemset, _halloc, _hfree, _lfind, _lsearch, _memccpy, memchr, _memicmp, memmove, qsort, memcmp, memcpy, memset. The last 3 of these also have intrinsic versions (the compiler generates inline code instead of calling a function), but the inline versions do not support huge pointers.



(My HTML formatting test. Disregarg the following text.
<B>bold</B>, <I>italic</I>,<PRE>
    preformatted text,    non roportional font
</PRE>
link to <A HREF="http://www.experts-exchange.com/Q.10121056">this page</A>.)
0
 

Author Comment

by:parlando
ID: 1258358
Thanks guys,
With this information I'll be able to solve the problem.I've learned a few new things about (huge) memory models and restrictions.
0

Featured Post

What Should I Do With This Threat Intelligence?

Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

Join & Write a Comment

An Outlet in Cocoa is a persistent reference to a GUI control; it connects a property (a variable) to a control.  For example, it is common to create an Outlet for the text field GUI control and change the text that appears in this field via that Ou…
This tutorial is posted by Aaron Wojnowski, administrator at SDKExpert.net.  To view more iPhone tutorials, visit www.sdkexpert.net. This is a very simple tutorial on finding the user's current location easily. In this tutorial, you will learn ho…
The goal of this video is to provide viewers with basic examples to understand opening and writing to files in the C programming language.
Video by: Grant
The goal of this video is to provide viewers with basic examples to understand and use for-loops in the C programming language.

758 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now