PowerShell: Find & Replace Hex Values

Member_2_8180951
Member_2_8180951 used Ask the Experts™
on
I need help writing a PowerShell script: write offset positions, find and replace hex value with another hex value in an entire file.

Similar to,
$Hex_Search = "0xFF 0xFF 0xFF" # White

Write-Host Found "$Hex_Search" Between Offsets: 0x00000500-0x00001000

$Hex_Replace = "0xC0 0xC0 0xC0" # Silver

Open in new window


For my script the file can be any size.
Been using two PowerShell scripts, no avail.
# Credit to OP

$path = "C:\_Test 2\Test.txt"

$numberOfBytesToRead = 1000000

$stringToSearch = "0xFF 0xFF 0xFF"
$enc = [system.Text.Encoding]::UTF8
[Byte[]]$replacementString = $enc.GetBytes("0xC0 0xC0 0xC0");

$fileStream = [System.IO.File]::Open($path, [System.IO.FileMode]::Open, [System.IO.FileAccess]::Read, [System.IO.FileShare]::ReadWrite)

# binary reader to search for the string 
$binaryReader = New-Object System.IO.BinaryReader($fileStream)

# get the contents of the beginning of the file
[Byte[]] $byteArray = $binaryReader.ReadBytes($numberOfBytesToRead)

# look for string
$m = [Regex]::Match([Text.Encoding]::ASCII.GetString($byteArray), $stringToSearch)
if ($m.Success)
{    
    echo "Found '$stringToSearch' At Position "$m.Index
}
else
{
    echo "'$stringToSearch' Not Found"
}

$fileStream.Close()

# reopen to write
$fileStream = [System.IO.File]::Open($path, [System.IO.FileMode]::Open, [System.IO.FileAccess]::Write, [System.IO.FileShare]::ReadWrite)

$binaryWriter = New-Object System.IO.BinaryWriter($fileStream)

# set file position to location of the string
$binaryWriter.BaseStream.Position = $m.Index; 
$binaryWriter.Write($replacementString)

$fileStream.Close()

Open in new window


(Get-Content "C:\_Test 2\Test.txt") | ForEach-Object {$_ -Replace '0xFF 0xFF 0xFF', '0xC0 0xC0 0xC0'} | Set-Content "C:\_Test 2\Test.txt"

Open in new window


Thanks In Advance
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Top Expert 2014

Commented:
Can you provide a couple sample files?
One in the format you're ingesting and that has the values you want to replace, and one that shows the file as it should be after processing.

This will be the easiest way to ensure there's no confusion around objectives, etc.
Michael B. SmithManaging Consultant

Commented:
I think you may be overthinking this. :-)

[byte[]] $searchBytes = 0x3, 0x4, 0x5  ## whatever
[byte[]] $content = Get-Content -Path <filename> -Raw -ReadCount 0 -Encoding Byte
if( $content.Contains( $searchBytes ) )
{
    $startIndex = $content.IndexOf( $searchBytes )
    Write-Host "Found searchBytes at $startIndex"
    ## well, replace it and rewrite it - you already know how to do that
}

Open in new window

Author

Commented:
I don't have example files (apologies), I need to find and replace hex (and print the offset range positions) similar to,
$Find_example = (0x65 0x78 0x61 0x6d 0x70 0x6c 0x65) # example (in hex)
Write-Host Found "$Find_example" Between Offsets: 0x00000500-0x00001000 # Just Example Position Range
$Replace_With_next = (0x6e 0x65 0x78 0x74) # next (in hex)

Open in new window


I actually need to find and replace color hex code,
$Find_Color_Hex = (0x00 0x80 0x00) # green in hex proper hex code (#008000)
Write-Host Found "$Find_Color_Hex" Between Offsets: 0x00000500-0x00001000 # Just Example Position Range
$Replace_Color_Hex = (0x87 0xCE 0xEB) # skyblue in hex proper hex code(#87CEEB)

Open in new window


@Michael B. Smith thanks for an answer, but the parameter "-Raw" isn't in PowerShell v2 ;_;
Introduction to R

R is considered the predominant language for data scientist and statisticians. Learn how to use R for your own data science projects.

Top Expert 2014

Commented:
The files should be pretty trivial for you to create.  You've already got files you want to work on (hence your question).  Just strip out all but a few lines that include the match you're searching for (you can obfuscate other info if needed).  Save it and you've got your sample input.  Do a manual edit to create your desired result, save it and you've got your sample output.
Michael B. SmithManaging Consultant

Commented:
Replace this line

[byte[] $content = Get-Content -Path <filename> -Raw -ReadCount 0 -Encoding Byte

with

[byte[] $content = [System.Io.File]::ReadAllBytes( <filename> )

To get (and I added the rest of it):

[byte[]] $searchBytes = 0x0, 0x80, 0x0  ## your example
[byte[]] $replaceBytes = 0x87, 0xCE, 0xEB  ## your example
[byte[]] $content = [System.Io.File]::ReadAllBytes( <filename> )
if( $content.Contains( $searchBytes ) )
{
    $startIndex = $content.IndexOf( $searchBytes )
    Write-Host "Found searchBytes at $startIndex"
    for( $i = 0; $i -lt $searchBytes.Length; $i++ )
    {
        $content[ $i + $startIndex ] = $replaceBytes[ $i ]
    }
    [System.Io.File]::WriteAllBytes( <filename>, $content )
}

Open in new window

Author

Commented:
Here is example in text document,
some hex values are: 0x49, 0x4B, 0x41 (IKA)
and need to change to: 0x6D, 0x75, 0x73, 0x75, 0x6D, 0x65 (musume)
and vice-versa

and print ranges, such as
0x00000000-0x00000002
0x00000053-0x00000055

@Michael B. Smith
I received error,
Error: "Method invocation failed because [System.Byte[] doesn't contain a method named 'Contains'." (Apologies, new to PowerShell.)

Edit: tried to upload text file? here is link to contents of Example.txt https://pastebin.com/pw1P9k2S
Michael B. SmithManaging Consultant

Commented:
How old a version of .NET and PowerShell are you running?

I copy-and-paste what I wrote above and it works.

Author

Commented:
PowerShell v2.0
.NET v3.0
Michael B. SmithManaging Consultant

Commented:
Word.

So the input and output lengths can be different?

Can there be multiple occurrences of the input string in the file?

Author

Commented:
Yes, input and output can be different lengths.

There can be more occurrences.
Managing Consultant
Commented:
Well this becomes far more difficult for old versions of PowerShell and old versions of .NET and when input/output are different lengths.

But nonetheless, here you go. In my example I take a TAB character and turn it into two spaces. My algorithm should work for any arbitrary source and destination byte strings.

##
## i'm not going to try to make these parameters on PSv2
##
[string] $file = 'c:\scripts\smm.ps1'

[byte[]] $searchBytes = 0x09
[byte[]] $replaceBytes = 0x20, 0x20
[byte[]] $content = [System.Io.File]::ReadAllBytes( $file )

function wv( $str )
{
    Write-Host $str
}

$script:matches = @()
function findmatches
{
    $length = $content.Length
    $index  = 0
    $stopIndex  = $length - $searchBytes.Length
    $searchLen  = $searchBytes.Length
    wv "findMatches: content.Length $( $content.Length ), stopIndex $stopIndex, searchLen $searchLen"
    do
    {
        if( $content[ $index ] -eq $searchBytes[ 0 ] )
        {
            ## got a potential hit
            $match = $true
            for( $i = 1; $i -lt $searchLen; $i++ )
            {
                if( $content[ $index + $i ] -ne $searchBytes[ $i ] )
                {
                    $match = $false
                    break
                }
            }
            if( $match )
            {
                wv "findmatches: found match at $index 0x$( $index.ToString('x') ) for $searchLen bytes"
                $script:matches += $index
                $index   += $searchLen
            }
            else
            {
                $index += 1
            }
        }
        else
        {
            $index += 1
        }
    } while ( $index -lt $stopIndex )

    wv "findmatches: found $( $script:matches.Count ) matches"
    return $script:matches.Count
}

function calculateResultLength
{
    $searchLen  = $searchBytes.Length
    $replaceLen = $replaceBytes.Length
    wv "calculateResultLength: searchLen $searchLen, replaceLen $replaceLen"

    if( $searchLen -eq $replaceLen )
    {
        return $content.Length
    }

    if( $replaceLen -lt $searchLen )
    {
        ## ain't worth calculating
        return $content.Length
    }

    $diff = $replaceLen - $searchLen
    $num  = $diff * $script:matches.Count

    $result = $content.Length + $num

    wv "calculateResultLength: diff $diff, num $num, result $result"
    return $result
}

$matchCount = findmatches
if( $matchCount -gt 0 )
{
    $destLength  = calculateResultLength
    $destContent = New-Object Byte[] $destLength

    $sourceIndex = 0 ## index into $content
    $destIndex   = 0 ## index into $destContent
    $matchIndex  = 0 ## index into $matches (based on the source, $content)

    $sourceLen   = $searchBytes.Length
    $destLen     = $replaceBytes.Length

    while( 1 )
    {
        $mIndex = $matches[ $matchIndex ]
        wv "moving to match $matchIndex at index $mIndex, sourceIndex $sourceIndex, destIndex $destIndex"
        while( $sourceIndex -lt $mIndex )
        {
            $destContent[ $destIndex ] = $content[ $sourceIndex ]
            $destIndex++
            $sourceIndex++
        }
        wv "sourceIndex now $sourceIndex, destIndex now $destIndex"
        for( $i = 0; $i -lt $destlen; $i++ )
        {
            $destContent[ $destIndex ] = $replaceBytes[ $i ]
            $destIndex++
        }
        wv "stored replacement, destIndex now $destIndex"
        $sourceIndex += $sourceLen
        $matchIndex++
        if( $matchIndex -eq $matchCount )
        {
            wv "processed all matches, sourceIndex $sourceIndex, destIndex $destIndex"
            break
        }
    }
    ## finish up any remainder after the last match
    while( $sourceIndex -lt $content.Length )
    {
        $destContent[ $destIndex ] = $content[ $sourceIndex ]
        $destIndex++
        $sourceIndex++
    }
    wv "all replacements complete, destIndex $destIndex, sourceIndex $sourceIndex"
    if( 0 ) ## change the 0 to 1, to update the file
    {
        [System.Io.File]::WriteAllBytes( $file, $destContent )
    }
    ## $global:destContent = $destContent
}
else
{
    wv "no matches, nothing will be changed"
}

Open in new window

Author

Commented:
Many Thanks @Michael B. Smith ^^

@footech if you have an solution please post it, I want learn from it :)
Top Expert 2014

Commented:
I've got nothing, as I was waiting on sample files.  I had some questions that the files would have definitively answered, much easier than my trying to type them out.

But no worries, I'm glad Michael's solution works for you.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial