Best Format To Use

Hi Experts,

     I have a spreadsheet with over 100k rows of data.  I typically have to search on 100 or more keywords within this spreadsheet.  What would be the best or fastest language to use for this if developing an app to perform this function?    Right now, the Visual Basic App that I have is taking over 20 minutes to search on just 60 words through 100k rows of data on my spreadsheet.  There's got to be a quicker way.  Any suggestions are very much appreciated.
itsmevicAsked:
Who is Participating?
 
broro183Commented:
hi,

Twenty minutes seems a long time. If you post your code we may be able to make some suggestions that will speed it up (eg remove any ".select", turn off calculation etc).


I've attached an example file with code that I wrote about 3 years ago (presented in: http://www.thecodecage.com/forumz/1054993832-post15.html ). The code is modified from Tushar's FindAll function. It is not polished, but it may give some ideas that can be included in your project.

On review of this old file, there are some things that I would write quite differently now, such as the "Public ranges" and the hardcoded "7" (bad Rob!) which resizes the search range to seven columns. The file uses ".find" which was fine for the 10-20k rows of data in the file's original usage. I'm not sure how it will perform with 100k rows of data, so it may not be as good an approach as the use of an "in memory (VBA) array".

Rob
1878d1329220888-spreadsheet-cont.xls
0
 
CodeCruiserCommented:
Would data remain in Excel? If yes then fastest option would be VBA.
0
 
clarkscottCommented:
Are you searching through every column in the rows... or just certain columns?  How many columns are you searching?  How much data is in these columns?

These are all important things to know to determine best methods.

Scott C
0
Get your problem seen by more experts

Be seen. Boost your question’s priority for more expert views and faster solutions

 
itsmevicAuthor Commented:
would be doing a complete sweep of the spreadsheet.  The column range would be from A to M.  Report size varies from 80k to 120k rows.  Data in these columns consists of single words, multiple words, file paths, domain\username, etc, etc....
0
 
itsmevicAuthor Commented:
Thank you!
0
 
broro183Commented:
hi itsmevic,

Thank you for the points. Would you mind showing the final code you are now using?

Rob
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.