Zocko2000
asked on
How to program a FAST CRC32 check
Dear Programmers,
I have a problem to calculate the CRC32 from a file. I have found a routine that can do this job (listing is at the end) but there is a small problem:
I have calculated the CRC32 with this program (compiled) and with other commercial programmes with a 3 mb big file.
Fact is that the commercial programmes had done the job nearly immediately and my program needs some seconds !!
I think this is too much time.
Could you give me a faster routine to get the CRC32 from a file ????
Thank you for your efforts
Michael
' ************************** ********** ********** ********** ********** ******
'// At top level of a module, always include to be sure that all variables have the right type
Option Compare Text
'// Then declare this array variable Crc32Table
Private Crc32Table(255) As Long
'// Then all we have to do is writing public functions like these...
Public Function InitCrc32(Optional ByVal Seed As Long = &HEDB88320, Optional ByVal Precondition As Long = &HFFFFFFFF) As Long
'// Declare counter variable iBytes, counter variable iBits, value variables lCrc32 and lTempCrc32
Dim iBytes As Integer, iBits As Integer, lCrc32 As Long, lTempCrc32 As Long
'// Turn on error trapping
On Error Resume Next
'// Iterate 256 times
For iBytes = 0 To 255
'// Initiate lCrc32 to counter variable
lCrc32 = iBytes
'// Now iterate through each bit in counter byte
For iBits = 0 To 7
'// Right shift unsigned long 1 bit
lTempCrc32 = lCrc32 And &HFFFFFFFE
lTempCrc32 = lTempCrc32 \ &H2
lTempCrc32 = lTempCrc32 And &H7FFFFFFF
'// Now check if temporary is less than zero and then mix Crc32 checksum with Seed value
If (lCrc32 And &H1) <> 0 Then
lCrc32 = lTempCrc32 Xor Seed
Else
lCrc32 = lTempCrc32
End If
Next
'// Put Crc32 checksum value in the holding array
Crc32Table(iBytes) = lCrc32
Next
'// After this is done, set function value to the precondition value
InitCrc32 = Precondition
End Function
'// The function above is the initializing function, now we have to write the computation function
Public Function AddCrc32(ByVal Item As String, ByVal Crc32 As Long) As Long
'// Declare following variables
Dim bCharValue As Byte, iCounter As Integer, lIndex As Long
Dim lAccValue As Long, lTableValue As Long
'// Turn on error trapping
On Error Resume Next
'// Iterate through the string that is to be checksum-computed
For iCounter = 1 To Len(Item)
'// Get ASCII value for the current character
bCharValue = Asc(Mid$(Item, iCounter, 1))
'// Right shift an Unsigned Long 8 bits
lAccValue = Crc32 And &HFFFFFF00
lAccValue = lAccValue \ &H100
lAccValue = lAccValue And &HFFFFFF
'// Now select the right adding value from the holding table
lIndex = Crc32 And &HFF
lIndex = lIndex Xor bCharValue
lTableValue = Crc32Table(lIndex)
'// Then mix new Crc32 value with previous accumulated Crc32 value
Crc32 = lAccValue Xor lTableValue
Next
'// Set function value the the new Crc32 checksum
AddCrc32 = Crc32
End Function
'// At last, we have to write a function so that we can get the Crc32 checksum value at any time
Public Function GetCrc32(ByVal Crc32 As Long) As Long
'// Turn on error trapping
On Error Resume Next
'// Set function to the current Crc32 value
GetCrc32 = Crc32 Xor &HFFFFFFFF
End Function
Private Sub Command1_Click()
Dim lCrc32Value As Long
On Error Resume Next
lCrc32Value = InitCrc32()
Open "c:\autoexec.bat" For Binary As #1
whole& = LOF(1) \ 20000
part& = LOF(1) Mod 20000
Original$ = String$(20000, 0)
Cracked$ = String$(20000, 0)
start& = 1
Dateigröße = LOF(1)
Do
x& = x& + 1
If x& <= whole& Then
Get #1, start&, Original$
Buffergröße = whole&
Else
Original$ = String$(part&, 0)
Get #1, start&, Original$
Buffergröße = part&
End If
' Call CRC32
lCrc32Value = AddCrc32(Original$, lCrc32Value)
a$ = Hex$(GetCrc32(lCrc32Value) )
Debug.Print a$
start& = start& + 20000
If start& >= Dateigröße Then Exit Do
Loop
Close
MsgBox "CRC32: " & a$
End Sub
I have a problem to calculate the CRC32 from a file. I have found a routine that can do this job (listing is at the end) but there is a small problem:
I have calculated the CRC32 with this program (compiled) and with other commercial programmes with a 3 mb big file.
Fact is that the commercial programmes had done the job nearly immediately and my program needs some seconds !!
I think this is too much time.
Could you give me a faster routine to get the CRC32 from a file ????
Thank you for your efforts
Michael
' **************************
'// At top level of a module, always include to be sure that all variables have the right type
Option Compare Text
'// Then declare this array variable Crc32Table
Private Crc32Table(255) As Long
'// Then all we have to do is writing public functions like these...
Public Function InitCrc32(Optional ByVal Seed As Long = &HEDB88320, Optional ByVal Precondition As Long = &HFFFFFFFF) As Long
'// Declare counter variable iBytes, counter variable iBits, value variables lCrc32 and lTempCrc32
Dim iBytes As Integer, iBits As Integer, lCrc32 As Long, lTempCrc32 As Long
'// Turn on error trapping
On Error Resume Next
'// Iterate 256 times
For iBytes = 0 To 255
'// Initiate lCrc32 to counter variable
lCrc32 = iBytes
'// Now iterate through each bit in counter byte
For iBits = 0 To 7
'// Right shift unsigned long 1 bit
lTempCrc32 = lCrc32 And &HFFFFFFFE
lTempCrc32 = lTempCrc32 \ &H2
lTempCrc32 = lTempCrc32 And &H7FFFFFFF
'// Now check if temporary is less than zero and then mix Crc32 checksum with Seed value
If (lCrc32 And &H1) <> 0 Then
lCrc32 = lTempCrc32 Xor Seed
Else
lCrc32 = lTempCrc32
End If
Next
'// Put Crc32 checksum value in the holding array
Crc32Table(iBytes) = lCrc32
Next
'// After this is done, set function value to the precondition value
InitCrc32 = Precondition
End Function
'// The function above is the initializing function, now we have to write the computation function
Public Function AddCrc32(ByVal Item As String, ByVal Crc32 As Long) As Long
'// Declare following variables
Dim bCharValue As Byte, iCounter As Integer, lIndex As Long
Dim lAccValue As Long, lTableValue As Long
'// Turn on error trapping
On Error Resume Next
'// Iterate through the string that is to be checksum-computed
For iCounter = 1 To Len(Item)
'// Get ASCII value for the current character
bCharValue = Asc(Mid$(Item, iCounter, 1))
'// Right shift an Unsigned Long 8 bits
lAccValue = Crc32 And &HFFFFFF00
lAccValue = lAccValue \ &H100
lAccValue = lAccValue And &HFFFFFF
'// Now select the right adding value from the holding table
lIndex = Crc32 And &HFF
lIndex = lIndex Xor bCharValue
lTableValue = Crc32Table(lIndex)
'// Then mix new Crc32 value with previous accumulated Crc32 value
Crc32 = lAccValue Xor lTableValue
Next
'// Set function value the the new Crc32 checksum
AddCrc32 = Crc32
End Function
'// At last, we have to write a function so that we can get the Crc32 checksum value at any time
Public Function GetCrc32(ByVal Crc32 As Long) As Long
'// Turn on error trapping
On Error Resume Next
'// Set function to the current Crc32 value
GetCrc32 = Crc32 Xor &HFFFFFFFF
End Function
Private Sub Command1_Click()
Dim lCrc32Value As Long
On Error Resume Next
lCrc32Value = InitCrc32()
Open "c:\autoexec.bat" For Binary As #1
whole& = LOF(1) \ 20000
part& = LOF(1) Mod 20000
Original$ = String$(20000, 0)
Cracked$ = String$(20000, 0)
start& = 1
Dateigröße = LOF(1)
Do
x& = x& + 1
If x& <= whole& Then
Get #1, start&, Original$
Buffergröße = whole&
Else
Original$ = String$(part&, 0)
Get #1, start&, Original$
Buffergröße = part&
End If
' Call CRC32
lCrc32Value = AddCrc32(Original$, lCrc32Value)
a$ = Hex$(GetCrc32(lCrc32Value)
Debug.Print a$
start& = start& + 20000
If start& >= Dateigröße Then Exit Do
Loop
Close
MsgBox "CRC32: " & a$
End Sub
Working with strings are VERY SLOW compared to bytes. Read the file data into a byte array instead of a string, this way you don't have to use for example the Mid$ and Asc (<- very slow function) functions.
When you compile it use the advanced settings (remove integer checks and array bound checks), those settings improved my own crc32 alrogithm a lot.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Just wanted to tell you that that was a very kewl OCX, chris_a. It was 9x faster than my VB code so I guess I'll be using that one instead. Just wanted to thank you for a great product! ;)
Just 9x faster, if you hide the control (progress bar) things should get better, was that v1.2 or the 1.5b (I am just finalising custom crc's)
I tested a 4.5Mb and I did hide the control, but now I see that hiding/showing the control does not have any compact on the performance.
The CRC type is the correct CRC32 (CRC32Fwd).
Your OCX takes 0.17 secs and my code takes 1.59 (9.35x)
The CRC type is the correct CRC32 (CRC32Fwd).
Your OCX takes 0.17 secs and my code takes 1.59 (9.35x)
Sorry my mistake a disk based file does limit things a bit, but 26MB/sec is not too shabby.
Oh forgot to say that, I read it into a byte array first, to eliminate any uneven disk performance.
Ouch, not just to calculate the CRC I hope, if so try the v1.5B, it does files on the disk async if required
ASKER
Hello Chris,
Thank you very much for your help. Your solution is much faster as I hoped for. Thank you for your help!
Michael
Thank you very much for your help. Your solution is much faster as I hoped for. Thank you for your help!
Michael