Link to home
Start Free TrialLog in
Avatar of Zocko2000
Zocko2000

asked on

How to program a FAST CRC32 check

Dear Programmers,

I have a problem to calculate the CRC32 from a file. I have found a routine that can do this job (listing is at the end) but there is a small problem:
I have calculated the CRC32 with this program (compiled) and with other commercial programmes with a 3 mb big file.
Fact is that the commercial programmes had done the job nearly immediately and my program needs some seconds !!
I think this is too much time.
Could you give me a faster routine to get the CRC32 from a file ????

Thank you for your efforts

Michael



' ************************************************************************

'// At top level of a module, always include to be sure that all variables have the right type
Option Compare Text

'// Then declare this array variable Crc32Table
Private Crc32Table(255) As Long

'// Then all we have to do is writing public functions like these...
Public Function InitCrc32(Optional ByVal Seed As Long = &HEDB88320, Optional ByVal Precondition As Long = &HFFFFFFFF) As Long

   '// Declare counter variable iBytes, counter variable iBits, value variables lCrc32 and lTempCrc32
   Dim iBytes As Integer, iBits As Integer, lCrc32 As Long, lTempCrc32 As Long

   '// Turn on error trapping
   On Error Resume Next

   '// Iterate 256 times
   For iBytes = 0 To 255

      '// Initiate lCrc32 to counter variable
      lCrc32 = iBytes

      '// Now iterate through each bit in counter byte
      For iBits = 0 To 7
         '// Right shift unsigned long 1 bit
         lTempCrc32 = lCrc32 And &HFFFFFFFE
         lTempCrc32 = lTempCrc32 \ &H2
         lTempCrc32 = lTempCrc32 And &H7FFFFFFF

         '// Now check if temporary is less than zero and then mix Crc32 checksum with Seed value
         If (lCrc32 And &H1) <> 0 Then
            lCrc32 = lTempCrc32 Xor Seed
         Else
            lCrc32 = lTempCrc32
         End If
      Next

      '// Put Crc32 checksum value in the holding array
      Crc32Table(iBytes) = lCrc32
   Next

   '// After this is done, set function value to the precondition value
   InitCrc32 = Precondition

End Function

'// The function above is the initializing function, now we have to write the computation function
Public Function AddCrc32(ByVal Item As String, ByVal Crc32 As Long) As Long

   '// Declare following variables
   Dim bCharValue As Byte, iCounter As Integer, lIndex As Long
   Dim lAccValue As Long, lTableValue As Long

   '// Turn on error trapping
   On Error Resume Next

   '// Iterate through the string that is to be checksum-computed
   For iCounter = 1 To Len(Item)

      '// Get ASCII value for the current character
      bCharValue = Asc(Mid$(Item, iCounter, 1))

      '// Right shift an Unsigned Long 8 bits
      lAccValue = Crc32 And &HFFFFFF00
      lAccValue = lAccValue \ &H100
      lAccValue = lAccValue And &HFFFFFF

      '// Now select the right adding value from the holding table
      lIndex = Crc32 And &HFF
      lIndex = lIndex Xor bCharValue
      lTableValue = Crc32Table(lIndex)

      '// Then mix new Crc32 value with previous accumulated Crc32 value
      Crc32 = lAccValue Xor lTableValue
   Next

   '// Set function value the the new Crc32 checksum
   AddCrc32 = Crc32

End Function

'// At last, we have to write a function so that we can get the Crc32 checksum value at any time
Public Function GetCrc32(ByVal Crc32 As Long) As Long
   '// Turn on error trapping
   On Error Resume Next

   '// Set function to the current Crc32 value
   GetCrc32 = Crc32 Xor &HFFFFFFFF

End Function




Private Sub Command1_Click()

  Dim lCrc32Value As Long
  On Error Resume Next
   
   
  lCrc32Value = InitCrc32()
   
   
   
   
  Open "c:\autoexec.bat" For Binary As #1
   
  whole& = LOF(1) \ 20000
  part& = LOF(1) Mod 20000
  Original$ = String$(20000, 0)
  Cracked$ = String$(20000, 0)
  start& = 1
  Dateigröße = LOF(1)
       
               
  Do
        x& = x& + 1
       
            If x& <= whole& Then
            Get #1, start&, Original$
            Buffergröße = whole&
                   
        Else
           
            Original$ = String$(part&, 0)
                     
            Get #1, start&, Original$
            Buffergröße = part&
             
        End If
               
               
           ' Call CRC32
           lCrc32Value = AddCrc32(Original$, lCrc32Value)
           a$ = Hex$(GetCrc32(lCrc32Value))
           Debug.Print a$
   
       
               start& = start& + 20000
       
               If start& >= Dateigröße Then Exit Do
  Loop
       
  Close
   
  MsgBox "CRC32: " & a$
 

End Sub


Avatar of Vbmaster
Vbmaster

Working with strings are VERY SLOW compared to bytes. Read the file data into a byte array instead of a string, this way you don't have to use for example the Mid$ and Asc (<- very slow function) functions.

When you compile it use the advanced settings (remove integer checks and array bound checks), those settings improved my own crc32 alrogithm a lot.

ASKER CERTIFIED SOLUTION
Avatar of chris_a
chris_a

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Just wanted to tell you that that was a very kewl OCX, chris_a. It was 9x faster than my VB code so I guess I'll be using that one instead. Just wanted to thank you for a great product! ;)
Just 9x faster, if you hide the control (progress bar) things should get better, was that v1.2 or the 1.5b (I am just finalising custom crc's)
I tested a 4.5Mb and I did hide the control, but now I see that hiding/showing the control does not have any compact on the performance.

The CRC type is the correct CRC32 (CRC32Fwd).

Your OCX takes 0.17 secs and my code takes 1.59  (9.35x)
Sorry my mistake a disk based file does limit things a bit, but 26MB/sec is not too shabby.

Oh forgot to say that, I read it into a byte array first, to eliminate any uneven disk performance.

Ouch, not just to calculate the CRC I hope, if so try the v1.5B, it does files on the disk async if required
Avatar of Zocko2000

ASKER

Hello Chris,

Thank you very much for your help. Your solution is much faster as I hoped for. Thank you for your help!

Michael