I want to search thousands of files for a fixed string. Case Insensitive, exact match.
Boyer Moore Horspool is the obvious thought and there is a nice looking bit of code by Angus Johnson at http://www.users.on.net/johnson/delphi/searches.html
This uses file mapping which I guess is cool but every file is mapped into memory separately according to its size. This seems like there must be a lot of memory reallocs going on behind the scenes .. one time I have a file of 42 bytes, the next 42mb.
Q1. Is filemapping the best way to go for searching large numbers of files of varying sizes
Q2. Is there a way of using and reusing a fixed say 64K buffer instead of asking for large amounts of memory, but still using file mapping
Q3. Am I correct in my assumption that file reading and memory mapping will be the bottleneck, and that therefore Boyer-Moore is as good as anything ?
All insights and mega fast code gratefully received