Solved

Find and Replace Stream with 0s

Posted on 2016-11-06
8
62 Views
Last Modified: 2016-11-17
Greetings Sir,

I want to search the contents of a stream in a binary file and replace all the bytes present in it with 0s. I tried to implement it by taking the first 10 bytes of the stream and searching them in the file, then if >1 matches are found, confirmed it by SHA1. But it seems to be bugged and slow. Can ya please tell me how to do it?
0
Comment
Question by:Abhishek Sharma
  • 3
  • 2
  • 2
  • +1
8 Comments
 
LVL 100

Expert Comment

by:mlmcc
ID: 41876615
Are you trying to create a stream of 0s?

mlmcc
0
 

Author Comment

by:Abhishek Sharma
ID: 41876694
I'm trying to replace all the data in b/w two offsets with 0s, i.e., if I had data = "ABCDEFGHIJKLM" and I give offset of 4 and size 2, then it should become like this "ABCD00GHIJKLM". Just the binary version of it.
0
 
LVL 37

Expert Comment

by:Geert Gruwez
ID: 41876743
What code have you got so far ?
0
U.S. Department of Agriculture and Acronis Access

With the new era of mobile computing, smartphones and tablets, wireless communications and cloud services, the USDA sought to take advantage of a mobilized workforce and the blurring lines between personal and corporate computing resources.

 
LVL 26

Expert Comment

by:Sinisa Vuk
ID: 41878295
I would use memory mapped file and search through it with CompareMem function, replace with CopyMemory.... This is possible because with memory mapped file - you get pointer to "memory" which is great for CompareMem.
0
 
LVL 37

Accepted Solution

by:
Geert Gruwez earned 500 total points
ID: 41879354
you'll have to open the file for read/write and read all the contents piece by piece and replace the bytes
i wrote something like this once, for in place byte updates in a file

here are some samples:
https://www.experts-exchange.com/questions/26386805/How-do-I-Write-a-String-at-the-very-top-of-a-Text-file.html
https://www.experts-exchange.com/questions/23347230/Need-make-Seek-function-for-text-file.html#a21429333
0
 

Author Comment

by:Abhishek Sharma
ID: 41884584
I ended up writing one code too, but it's giving StackOverflow on big scan inputs:

program Project2;

{$APPTYPE CONSOLE}
{$R *.res}
{$optimization on}

uses
  WinAPI.Windows,
  System.SysUtils,
  System.Classes,
  System.Math;

type
  TByteArray = Array of Byte;

function StreamToByteArray(Stream: TStream): TByteArray;
begin
  if Assigned(Stream) then
  begin
    Stream.Position := 0;
    SetLength(result, Stream.Size);
    Stream.Read(result[0], Stream.Size);
  end
  else
    SetLength(result, 0);
end;

procedure FindHeader(FileName: string; Sequence: array of Byte;
  var List: TStringList);
var
  Buffer: array of Byte;
  Fs: TBufferedFileStream;
  l, n, sl: Int64;
  ScanStopped: boolean;

begin
  List.Clear;
  sl := length(Sequence);
  if sl = 0 then
    Exit;

  Fs := TBufferedFileStream.Create(FileName, fmOpenRead or fmShareDenyWrite,
    4 * 1024 * 1024);
  try
    SetLength(Buffer, Fs.Size);
    Fs.Read(Buffer[0], Fs.Size);
    for l := 0 to Fs.Size - 1 do
    begin
      if Buffer[l] = Sequence[0] then
      begin
        if sl = 1 then
          List.Add(IntToStr(l))
        else
        begin
          n := 1;
          ScanStopped := false;
          while n < sl do
          begin
            if Buffer[l + n] <> Sequence[n] then
            begin
              ScanStopped := true;
              Break;
            end;
            Inc(n);
          end;
          if not ScanStopped then
            List.Add(IntToStr(l));
        end;
      end;
    end;
  finally
    Finalize(Buffer);
    Fs.Free;
  end;
end;

var
  raw: TMemoryStream;
  null: array [0 .. 0] of Byte = (
    $00
  );
  pos: TStringList;
  I, j: Int64;
  start, stop, elapsed: cardinal;
  infile, scanfile: TBufferedFileStream;

begin
  start := GetTickCount;
  pos := TStringList.Create;
  j := 0;
  try
    scanfile := TBufferedFileStream.Create(paramstr(2), fmOpenReadWrite);
    raw := TMemoryStream.Create;
    if scanfile.Size > 1024 * 1024 then
      raw.CopyFrom(scanfile, 1024 * 1024)
    else
      raw.CopyFrom(scanfile, 0);
    scanfile.Free;
    FindHeader(paramstr(1), StreamToByteArray(raw), pos);
    writeln('Possible offsets found: ' + IntToStr(pos.Count));
    infile := TBufferedFileStream.Create(paramstr(1), fmOpenReadWrite);
    if pos.Count <> 0 then
    begin
      infile.Seek(strtoint(pos[0]), soFromBeginning);
      for I := 0 to raw.Size - 1 do
      begin
        infile.Write(null, 1);
      end;
    end;
  finally
    pos.Free;
    raw.Free;
    infile.Free;
  end;
  stop := GetTickCount;
  elapsed := stop - start;
  writeln('Time Taken: ' + IntToStr(elapsed) + ' ms');

end.

Open in new window

0
 
LVL 37

Expert Comment

by:Geert Gruwez
ID: 41885559
well yeah, you put the whole stream in memory with this function StreamToByteArray
why ?

read a chunk, analyze it for the items, update the positions on file, read next check,
repeat until Jezus Christ returns or EOF
0
 
LVL 26

Expert Comment

by:Sinisa Vuk
ID: 41891622
Try my piece of code for finding first position (with a replacing):
function MMFReplace(FileName: String; pFindWhat: TByteDynArray; pReplaceWith: TByteDynArray): Boolean;

  function MakeInt64 (H,L: DWord): Int64;
  begin
    Result := H or (L shl 32);
  end;

var
  hFile: THandle;
  hFileMap: THandle;
  hiSize, loSize: DWORD;
  pFileView: Pointer;
  lFileSz, lFind, lReplace: Int64;
  i: Cardinal;
begin
  Result := False;
  if FileName = '' then Exit;

  lFind := Length(pFindWhat);
  lReplace := Length(pReplaceWith);
  //check if parameter are same length
  //if Length(pFindWhat) <> Length(pReplaceWith) then Exit;

  //open the file...
  hFile := CreateFile(
    PChar(FileName), GENERIC_READ or GENERIC_WRITE, FILE_SHARE_READ or FILE_SHARE_WRITE, nil,
    OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);

  if hFile <> INVALID_HANDLE_VALUE then
  begin
    try
      loSize := GetFileSize(hFile, @hiSize);
      lFileSz := MakeInt64(hiSize, loSize);

      //create file mapping....
      hFileMap := CreateFileMapping(hFile, nil, PAGE_READWRITE, hiSize, loSize,  'Find_And_Replace_MMF');

      if hFileMap <> 0 then
      begin
        try
          //create map view....
          pFileView := MapViewOfFile(hFileMap, FILE_MAP_READ or FILE_MAP_WRITE, 0, 0, 0);
          if pFileView <> nil then
          begin
            try
              //try to find ..
              for i := 0 to lFileSz - lFind - 1 do
              begin
                //compare bytes...
                if CompareMem(Pointer(Cardinal(pFileView)+i), @pFindWhat[0], lFind) then
                begin
                  //found it... replace
                  if lReplace>0 then
                    MoveMemory(Pointer(Cardinal(pFileView)+i), @pReplaceWith[0], lReplace);
                  Result := True;
                  Break;
                end;
              end;
            finally
              UnmapViewOfFile(pFileView);
            end;
          end;
        finally
          CloseHandle(hFileMap);
        end;
      end;
    finally
      CloseHandle(hFile);
    end;
  end;
end;

procedure TForm1.Button1Click(Sender: TObject);
var
  sFind, sReplace: TByteDynArray;
begin
  SetLength(sFind, 4);
  sFind[0] := Ord('5');
  sFind[1] := Ord('0');
  sFind[2] := Ord('0');
  sFind[3] := Ord('2');

  SetLength(sReplace, 4);
  sReplace[0] := Ord('5');
  sReplace[1] := Ord('0');
  sReplace[2] := Ord('0');
  sReplace[3] := Ord('3');

  if MMFReplace('C:\Temp\myfile.txt', sFind, sReplace) then
    ShowMessage('Found!');
end;

Open in new window

0

Featured Post

Master Your Team's Linux and Cloud Stack!

The average business loses $13.5M per year to ineffective training (per 1,000 employees). Keep ahead of the competition and combine in-person quality with online cost and flexibility by training with Linux Academy.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
error 1.1 400 Bad request idhttp delphi 18 91
Strange behavior when a form is closed 6 58
LAN or WAN ? 11 92
tidtcpserver connection lost handle 2 87
Have you ever had your Delphi form/application just hanging while waiting for data to load? This is the article to read if you want to learn some things about adding threads for data loading in the background. First, I'll setup a general applica…
In my programming career I have only very rarely run into situations where operator overloading would be of any use in my work.  Normally those situations involved math with either overly large numbers (hundreds of thousands of digits or accuracy re…
This tutorial gives a high-level tour of the interface of Marketo (a marketing automation tool to help businesses track and engage prospective customers and drive them to purchase). You will see the main areas including Marketing Activities, Design …
This Micro Tutorial demonstrates using Microsoft Excel pivot tables, how to reverse engineer competitors' marketing strategies through backlinks.

809 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question