Typecasting dynamic length raw data

Using Delphi 7:

Simple situation, I want to typecast raw data using a buffer pointer and buffer length. I'm mainly looking for chars though it will be intermingled with other types. Since #0 termination byte may be in raw data, typecasting to PChar is unreliable, is there any other way to do this without iterating byte by byte. The buffer ranges from bytes to megabytes so I'm looking for something fast.

procedure GimmeData(pBuf: pointer; iBuffLen: word);
Type
 TBufData = array of byte;
 PTBufData = ^TBufData;
Var
 BufData: PTBufData;
...
Begin
 BufData := pBuf;

output(PChar(BufData^));//if only it were that easy

End


LVL 2
JebtrixAsked:
Who is Participating?
 
Russell LibbySoftware Engineer, Advisory Commented:
The delphi help goes into fairly good detail on dynamic arrays, eg those that are defined as:

type
  YourType = Array of DataType;

but is badly lacking in the mentioning of dyamically allocated arrays (ie pointer arrays), eg those that are defined as:

type
  PYourType = ^TYourType;
  TYourType = Array [0..DesiredMaxLimit] of DataType;

A good explanation of the second can be found at:

http://community.borland.com/article/0,1410,16093,00.html

But the point I was getting at is that you can't cast a raw block of memory as the first type, as delphi's dynamic arrays are length and ref counted (just like long strings). So, its better to stick with one or the other unless you are familiar with the nuances of each. If you ARE familiar with both, and know what you are doing, you can cast the address of element [0] of the first type as the second type, eg)

type
  PTData         =  ^TData; // You can use this to cast raw pointers with
  TData          =  Array [0..Pred(MaxInt)] of Byte; // Dont ever declare a variable of this type, it will consume 2GB of memory
                                                                       // it only serves as a means to declare a pointer type from it.
  TDataArray     =  Array of Byte; // You should not use this to cast raw pointers with

var  DataDest:      PTData;
     DataArray:     TDataArray;
begin

  SetLength(DataArray, 100);
  DataDest:=@DataArray[0];
  ....

But its still not ok to cast the second type (or raw pointer) to the first type, and the compiler will usually flag this error for you, as it did in your example.

Russell


0
 
Russell LibbySoftware Engineer, Advisory Commented:
Actually, it is that easy.

function GimmeData(Buffer: Pointer; Size: Integer): String;
var  s:    String;
begin
  SetString(result, PChar(Buffer), Size);
End;

procedure TForm1.Button1Click(Sender: TObject);
var  p:    Pointer;
     s:    String;
begin

  p:=AllocMem(1000);
  s:=GimmeData(p, 1000);
  if Length(s) = 1000 then
     beep

end;
0
 
JebtrixAuthor Commented:
Thanks rllibby, another case of RTFM on my part. Good this is only a hobby. But since I'm on this subject I did at one point try to copy the contents of the buffer into an array and it kept giving me this error I can't figure out. Encore please :)

Type
  TData = array of byte;
  PTData = ^TData;
Var
  DataDest: PTData;
  DataSrc : Pointer;

Begin
 DataSrc := allocmem(1000);
 new(DataDest);
 SetLength(TData(DataDest),100);
 DataDest := copy(TData(DataSrc),0,100);//[Error] incompatible types: 'Array' and 'TBufData'

 ...
 Dispose(BufData);
 FreeMem(temp);
0
Cloud Class® Course: Microsoft Office 2010

This course will introduce you to the interfaces and features of Microsoft Office 2010 Word, Excel, PowerPoint, Outlook, and Access. You will learn about the features that are shared between all products in the Office suite, as well as the new features that are product specific.

 
JebtrixAuthor Commented:
Correction:
//[Error] incompatible types: 'Array' and 'TData'
0
 
Russell LibbySoftware Engineer, Advisory Commented:

Ok... don't mix (or confuse) delphi dynamic arrays with memory that is type cast as an array of a data type, to be shown in a second. In the example below, you will see that a block of memory can be cast with my PTData declaration, but should not be cast using a type defined as Array of Byte. These arrays (dynamic arrays) are special in that delphi allocates memory before the address of element zero to store the length of the array as well as other bits of information used to handle the dynamic array.

Regards,
Russell

type
  PTData         =  ^TData; // You can use this to cast raw pointers with
  TData          =  Array [0..Pred(MaxInt)] of Byte; // Dont ever declare a variable of this type, it will consume 2GB of memory
                                                                       // it only serves as a means to declare a pointer type from it.
  TDataArray     =  Array of Byte; // You should not use this to cast raw pointers with

procedure Test;
var  DataDest:      PTData;
     DataArray:     TDataArray;
     DataSrc:       Pointer;
     dwIndex:       Integer;
begin

  // Create source buffer
  DataSrc:=AllocMem(1000);

  try
     // Fill in source with some values
     for dwIndex:=0 to 1000 do PTData(DataSrc)^[dwIndex]:=dwIndex;
     // Allocate memory that is type cast as byte array (note, this is not the same array of byte dynamic array)
     DataDest:=AllocMem(100);
     try
        // Move data from source to dest
        Move(DataSrc^, DataDest^, 100);
        // Or using a dynamic array
        SetLength(DataArray, 100);
        // Move data to dynamic array
        Move(DataSrc^, DataArray[0], 100);
     finally
        FreeMem(DataDest);
     end;
  finally
     FreeMem(DataSrc);
  end;

end;


0
 
Wim ten BrinkSelf-employed developerCommented:
Use SysUtils and use the PByteArray/TByteArray types... Don't use your own dynamic array.

Then again, if you do want to use dynamic arrays, do it like this:
procedure GimmeData(pBuf: pointer; iBuffLen: word);
Type
 TBufData = array of byte;
Var
 BufData: TTBufData;
...
Begin
  SetLength(BufData, iBuffLen);
  Move(pBuf, Pointer(BufData)^, iBuffLen)
  ...

A dynamic array is like a string. It's basically just a special kind of pointer which happens to know the amount of memory that it needs.

But rlibby did great at explaining this. ;-) I'm just summing it all up.
0
 
Russell LibbySoftware Engineer, Advisory Commented:
>> Don't use your own dynamic array

I would be almost willing to agree EXCEPT for the fact that Borland declared it as

  PByteArray = ^TByteArray;
  TByteArray = array[0..32767] of Byte;

A little too short for what most people use it for. And the example above is incorrect, it should have been:

Type
 TBufData = array of byte;
Var
 BufData: TBufData; // not TTBufData
begin
  SetLength(BufData, iBuffLen);
  Move(pBuf^, Pointer(BufData)^, iBuffLen); // pBuf needs to be dereferenced

----

Russell



0
 
Wim ten BrinkSelf-employed developerCommented:
Russell,

While Borland defined it as 0..32767 the reality is that if you turn off the range checking compiler option then Delphi will just ignore it's fixed length. Only with range checking enabled will you get problems if you have more than 32768 elements. In most cases you will have considerably less so this structure should even be okay.

You were right about the pBuf, btw. But you could also use this:
procedure GimmeData(var pBuf; iBuffLen: word);

The last thing is what I prefer to use in most cases anyways.
0
 
Russell LibbySoftware Engineer, Advisory Commented:

I realize this fact, but I am not the one asking the question, and you should have stated your personal preference (because thats what it is) instead of a blanket statement:

>>  Don't use your own dynamic array

with nothing else to go along with it.

0
 
JebtrixAuthor Commented:
Awesome answers, crystal clear now.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.