Solved

Typecasting dynamic length raw data

Posted on 2006-07-17
10
574 Views
Last Modified: 2010-04-04
Using Delphi 7:

Simple situation, I want to typecast raw data using a buffer pointer and buffer length. I'm mainly looking for chars though it will be intermingled with other types. Since #0 termination byte may be in raw data, typecasting to PChar is unreliable, is there any other way to do this without iterating byte by byte. The buffer ranges from bytes to megabytes so I'm looking for something fast.

procedure GimmeData(pBuf: pointer; iBuffLen: word);
Type
 TBufData = array of byte;
 PTBufData = ^TBufData;
Var
 BufData: PTBufData;
...
Begin
 BufData := pBuf;

output(PChar(BufData^));//if only it were that easy

End


0
Comment
Question by:Jebtrix
  • 5
  • 3
  • 2
10 Comments
 
LVL 26

Expert Comment

by:Russell Libby
ID: 17125912
Actually, it is that easy.

function GimmeData(Buffer: Pointer; Size: Integer): String;
var  s:    String;
begin
  SetString(result, PChar(Buffer), Size);
End;

procedure TForm1.Button1Click(Sender: TObject);
var  p:    Pointer;
     s:    String;
begin

  p:=AllocMem(1000);
  s:=GimmeData(p, 1000);
  if Length(s) = 1000 then
     beep

end;
0
 
LVL 2

Author Comment

by:Jebtrix
ID: 17127628
Thanks rllibby, another case of RTFM on my part. Good this is only a hobby. But since I'm on this subject I did at one point try to copy the contents of the buffer into an array and it kept giving me this error I can't figure out. Encore please :)

Type
  TData = array of byte;
  PTData = ^TData;
Var
  DataDest: PTData;
  DataSrc : Pointer;

Begin
 DataSrc := allocmem(1000);
 new(DataDest);
 SetLength(TData(DataDest),100);
 DataDest := copy(TData(DataSrc),0,100);//[Error] incompatible types: 'Array' and 'TBufData'

 ...
 Dispose(BufData);
 FreeMem(temp);
0
 
LVL 2

Author Comment

by:Jebtrix
ID: 17127641
Correction:
//[Error] incompatible types: 'Array' and 'TData'
0
 
LVL 26

Expert Comment

by:Russell Libby
ID: 17127683

Ok... don't mix (or confuse) delphi dynamic arrays with memory that is type cast as an array of a data type, to be shown in a second. In the example below, you will see that a block of memory can be cast with my PTData declaration, but should not be cast using a type defined as Array of Byte. These arrays (dynamic arrays) are special in that delphi allocates memory before the address of element zero to store the length of the array as well as other bits of information used to handle the dynamic array.

Regards,
Russell

type
  PTData         =  ^TData; // You can use this to cast raw pointers with
  TData          =  Array [0..Pred(MaxInt)] of Byte; // Dont ever declare a variable of this type, it will consume 2GB of memory
                                                                       // it only serves as a means to declare a pointer type from it.
  TDataArray     =  Array of Byte; // You should not use this to cast raw pointers with

procedure Test;
var  DataDest:      PTData;
     DataArray:     TDataArray;
     DataSrc:       Pointer;
     dwIndex:       Integer;
begin

  // Create source buffer
  DataSrc:=AllocMem(1000);

  try
     // Fill in source with some values
     for dwIndex:=0 to 1000 do PTData(DataSrc)^[dwIndex]:=dwIndex;
     // Allocate memory that is type cast as byte array (note, this is not the same array of byte dynamic array)
     DataDest:=AllocMem(100);
     try
        // Move data from source to dest
        Move(DataSrc^, DataDest^, 100);
        // Or using a dynamic array
        SetLength(DataArray, 100);
        // Move data to dynamic array
        Move(DataSrc^, DataArray[0], 100);
     finally
        FreeMem(DataDest);
     end;
  finally
     FreeMem(DataSrc);
  end;

end;


0
 
LVL 26

Accepted Solution

by:
Russell Libby earned 90 total points
ID: 17127768
The delphi help goes into fairly good detail on dynamic arrays, eg those that are defined as:

type
  YourType = Array of DataType;

but is badly lacking in the mentioning of dyamically allocated arrays (ie pointer arrays), eg those that are defined as:

type
  PYourType = ^TYourType;
  TYourType = Array [0..DesiredMaxLimit] of DataType;

A good explanation of the second can be found at:

http://community.borland.com/article/0,1410,16093,00.html

But the point I was getting at is that you can't cast a raw block of memory as the first type, as delphi's dynamic arrays are length and ref counted (just like long strings). So, its better to stick with one or the other unless you are familiar with the nuances of each. If you ARE familiar with both, and know what you are doing, you can cast the address of element [0] of the first type as the second type, eg)

type
  PTData         =  ^TData; // You can use this to cast raw pointers with
  TData          =  Array [0..Pred(MaxInt)] of Byte; // Dont ever declare a variable of this type, it will consume 2GB of memory
                                                                       // it only serves as a means to declare a pointer type from it.
  TDataArray     =  Array of Byte; // You should not use this to cast raw pointers with

var  DataDest:      PTData;
     DataArray:     TDataArray;
begin

  SetLength(DataArray, 100);
  DataDest:=@DataArray[0];
  ....

But its still not ok to cast the second type (or raw pointer) to the first type, and the compiler will usually flag this error for you, as it did in your example.

Russell


0
IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 
LVL 17

Expert Comment

by:Wim ten Brink
ID: 17128338
Use SysUtils and use the PByteArray/TByteArray types... Don't use your own dynamic array.

Then again, if you do want to use dynamic arrays, do it like this:
procedure GimmeData(pBuf: pointer; iBuffLen: word);
Type
 TBufData = array of byte;
Var
 BufData: TTBufData;
...
Begin
  SetLength(BufData, iBuffLen);
  Move(pBuf, Pointer(BufData)^, iBuffLen)
  ...

A dynamic array is like a string. It's basically just a special kind of pointer which happens to know the amount of memory that it needs.

But rlibby did great at explaining this. ;-) I'm just summing it all up.
0
 
LVL 26

Expert Comment

by:Russell Libby
ID: 17129974
>> Don't use your own dynamic array

I would be almost willing to agree EXCEPT for the fact that Borland declared it as

  PByteArray = ^TByteArray;
  TByteArray = array[0..32767] of Byte;

A little too short for what most people use it for. And the example above is incorrect, it should have been:

Type
 TBufData = array of byte;
Var
 BufData: TBufData; // not TTBufData
begin
  SetLength(BufData, iBuffLen);
  Move(pBuf^, Pointer(BufData)^, iBuffLen); // pBuf needs to be dereferenced

----

Russell



0
 
LVL 17

Expert Comment

by:Wim ten Brink
ID: 17130229
Russell,

While Borland defined it as 0..32767 the reality is that if you turn off the range checking compiler option then Delphi will just ignore it's fixed length. Only with range checking enabled will you get problems if you have more than 32768 elements. In most cases you will have considerably less so this structure should even be okay.

You were right about the pBuf, btw. But you could also use this:
procedure GimmeData(var pBuf; iBuffLen: word);

The last thing is what I prefer to use in most cases anyways.
0
 
LVL 26

Expert Comment

by:Russell Libby
ID: 17130489

I realize this fact, but I am not the one asking the question, and you should have stated your personal preference (because thats what it is) instead of a blanket statement:

>>  Don't use your own dynamic array

with nothing else to go along with it.

0
 
LVL 2

Author Comment

by:Jebtrix
ID: 17131162
Awesome answers, crystal clear now.
0

Featured Post

Why You Should Analyze Threat Actor TTPs

After years of analyzing threat actor behavior, it’s become clear that at any given time there are specific tactics, techniques, and procedures (TTPs) that are particularly prevalent. By analyzing and understanding these TTPs, you can dramatically enhance your security program.

Join & Write a Comment

Introduction The parallel port is a very commonly known port, it was widely used to connect a printer to the PC, if you look at the back of your computer, for those who don't have newer computers, there will be a port with 25 pins and a small print…
Introduction Raise your hands if you were as upset with FireMonkey as I was when I discovered that there was no TListview.  I use TListView in almost all of my applications I've written, and I was not going to compromise by resorting to TStringGrid…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…
This tutorial demonstrates a quick way of adding group price to multiple Magento products.

760 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

22 Experts available now in Live!

Get 1:1 Help Now