Solved

Best Format To Use

Posted on 2012-12-28
6
157 Views
Last Modified: 2013-01-10
Hi Experts,

     I have a spreadsheet with over 100k rows of data.  I typically have to search on 100 or more keywords within this spreadsheet.  What would be the best or fastest language to use for this if developing an app to perform this function?    Right now, the Visual Basic App that I have is taking over 20 minutes to search on just 60 words through 100k rows of data on my spreadsheet.  There's got to be a quicker way.  Any suggestions are very much appreciated.
0
Comment
Question by:itsmevic
6 Comments
 
LVL 83

Assisted Solution

by:CodeCruiser
CodeCruiser earned 250 total points
ID: 38728185
Would data remain in Excel? If yes then fastest option would be VBA.
0
 
LVL 10

Accepted Solution

by:
broro183 earned 250 total points
ID: 38730305
hi,

Twenty minutes seems a long time. If you post your code we may be able to make some suggestions that will speed it up (eg remove any ".select", turn off calculation etc).


I've attached an example file with code that I wrote about 3 years ago (presented in: http://www.thecodecage.com/forumz/1054993832-post15.html ). The code is modified from Tushar's FindAll function. It is not polished, but it may give some ideas that can be included in your project.

On review of this old file, there are some things that I would write quite differently now, such as the "Public ranges" and the hardcoded "7" (bad Rob!) which resizes the search range to seven columns. The file uses ".find" which was fine for the 10-20k rows of data in the file's original usage. I'm not sure how it will perform with 100k rows of data, so it may not be as good an approach as the use of an "in memory (VBA) array".

Rob
1878d1329220888-spreadsheet-cont.xls
0
 
LVL 20

Expert Comment

by:clarkscott
ID: 38734328
Are you searching through every column in the rows... or just certain columns?  How many columns are you searching?  How much data is in these columns?

These are all important things to know to determine best methods.

Scott C
0
Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

 

Author Comment

by:itsmevic
ID: 38741334
would be doing a complete sweep of the spreadsheet.  The column range would be from A to M.  Report size varies from 80k to 120k rows.  Data in these columns consists of single words, multiple words, file paths, domain\username, etc, etc....
0
 

Author Closing Comment

by:itsmevic
ID: 38761254
Thank you!
0
 
LVL 10

Expert Comment

by:broro183
ID: 38762427
hi itsmevic,

Thank you for the points. Would you mind showing the final code you are now using?

Rob
0

Featured Post

Free Tool: Subnet Calculator

The subnet calculator helps you design networks by taking an IP address and network mask and returning information such as network, broadcast address, and host range.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Workbook link problems after copying tabs to a new workbook? David Miller (dlmille) Intro Have you either copied sheets to a new workbook, and after having saved and opened that workbook, you find that there are links back to the original sou…
Not long ago I saw a question in the VB Script forum that I thought would not take much time. You can read that question (Question ID  (http://www.experts-exchange.com/Programming/Languages/Visual_Basic/VB_Script/Q_28455246.html)28455246) Here (http…
This Micro Tutorial demonstrates using Microsoft Excel pivot tables, how to reverse engineer competitors' marketing strategies through backlinks.
This Micro Tutorial demonstrates in Microsoft Excel how to consolidate your marketing data by creating an interactive charts using form controls. This creates cool drop-downs for viewers of your chart to choose from.

856 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question