[Okta Webinar] Learn how to a build a cloud-first strategyRegister Now

x
?
Solved

Best Format To Use

Posted on 2012-12-28
6
Medium Priority
?
174 Views
Last Modified: 2013-01-10
Hi Experts,

     I have a spreadsheet with over 100k rows of data.  I typically have to search on 100 or more keywords within this spreadsheet.  What would be the best or fastest language to use for this if developing an app to perform this function?    Right now, the Visual Basic App that I have is taking over 20 minutes to search on just 60 words through 100k rows of data on my spreadsheet.  There's got to be a quicker way.  Any suggestions are very much appreciated.
0
Comment
Question by:itsmevic
6 Comments
 
LVL 83

Assisted Solution

by:CodeCruiser
CodeCruiser earned 1000 total points
ID: 38728185
Would data remain in Excel? If yes then fastest option would be VBA.
0
 
LVL 10

Accepted Solution

by:
broro183 earned 1000 total points
ID: 38730305
hi,

Twenty minutes seems a long time. If you post your code we may be able to make some suggestions that will speed it up (eg remove any ".select", turn off calculation etc).


I've attached an example file with code that I wrote about 3 years ago (presented in: http://www.thecodecage.com/forumz/1054993832-post15.html ). The code is modified from Tushar's FindAll function. It is not polished, but it may give some ideas that can be included in your project.

On review of this old file, there are some things that I would write quite differently now, such as the "Public ranges" and the hardcoded "7" (bad Rob!) which resizes the search range to seven columns. The file uses ".find" which was fine for the 10-20k rows of data in the file's original usage. I'm not sure how it will perform with 100k rows of data, so it may not be as good an approach as the use of an "in memory (VBA) array".

Rob
1878d1329220888-spreadsheet-cont.xls
0
 
LVL 20

Expert Comment

by:clarkscott
ID: 38734328
Are you searching through every column in the rows... or just certain columns?  How many columns are you searching?  How much data is in these columns?

These are all important things to know to determine best methods.

Scott C
0
What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

 

Author Comment

by:itsmevic
ID: 38741334
would be doing a complete sweep of the spreadsheet.  The column range would be from A to M.  Report size varies from 80k to 120k rows.  Data in these columns consists of single words, multiple words, file paths, domain\username, etc, etc....
0
 

Author Closing Comment

by:itsmevic
ID: 38761254
Thank you!
0
 
LVL 10

Expert Comment

by:broro183
ID: 38762427
hi itsmevic,

Thank you for the points. Would you mind showing the final code you are now using?

Rob
0

Featured Post

VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

When you see single cell contains number and text, and you have to get any date out of it seems like cracking our heads.
Access developers frequently have requirements to interact with Excel (import from or output to) in their applications.  You might be able to accomplish this with the TransferSpreadsheet and OutputTo methods, but in this series of articles I will di…
This Micro Tutorial will demonstrate on a Mac how to change the sort order for chart legend values and decrpyt the intimidating chart menu.
This Micro Tutorial will demonstrate how to create pivot charts out of a data set. I also added a drop-down menu which allows to choose from different categories in the data set and the chart will automatically update.

834 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question