?
Solved

Best Format To Use

Posted on 2012-12-28
6
Medium Priority
?
171 Views
Last Modified: 2013-01-10
Hi Experts,

     I have a spreadsheet with over 100k rows of data.  I typically have to search on 100 or more keywords within this spreadsheet.  What would be the best or fastest language to use for this if developing an app to perform this function?    Right now, the Visual Basic App that I have is taking over 20 minutes to search on just 60 words through 100k rows of data on my spreadsheet.  There's got to be a quicker way.  Any suggestions are very much appreciated.
0
Comment
Question by:itsmevic
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
6 Comments
 
LVL 83

Assisted Solution

by:CodeCruiser
CodeCruiser earned 1000 total points
ID: 38728185
Would data remain in Excel? If yes then fastest option would be VBA.
0
 
LVL 10

Accepted Solution

by:
broro183 earned 1000 total points
ID: 38730305
hi,

Twenty minutes seems a long time. If you post your code we may be able to make some suggestions that will speed it up (eg remove any ".select", turn off calculation etc).


I've attached an example file with code that I wrote about 3 years ago (presented in: http://www.thecodecage.com/forumz/1054993832-post15.html ). The code is modified from Tushar's FindAll function. It is not polished, but it may give some ideas that can be included in your project.

On review of this old file, there are some things that I would write quite differently now, such as the "Public ranges" and the hardcoded "7" (bad Rob!) which resizes the search range to seven columns. The file uses ".find" which was fine for the 10-20k rows of data in the file's original usage. I'm not sure how it will perform with 100k rows of data, so it may not be as good an approach as the use of an "in memory (VBA) array".

Rob
1878d1329220888-spreadsheet-cont.xls
0
 
LVL 20

Expert Comment

by:clarkscott
ID: 38734328
Are you searching through every column in the rows... or just certain columns?  How many columns are you searching?  How much data is in these columns?

These are all important things to know to determine best methods.

Scott C
0
What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

 

Author Comment

by:itsmevic
ID: 38741334
would be doing a complete sweep of the spreadsheet.  The column range would be from A to M.  Report size varies from 80k to 120k rows.  Data in these columns consists of single words, multiple words, file paths, domain\username, etc, etc....
0
 

Author Closing Comment

by:itsmevic
ID: 38761254
Thank you!
0
 
LVL 10

Expert Comment

by:broro183
ID: 38762427
hi itsmevic,

Thank you for the points. Would you mind showing the final code you are now using?

Rob
0

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article descibes how to create a connection between Excel and SAP and how to move data from Excel to SAP or the other way around.
Excel can be a tricky bit of software to get your head around. Whilst you’ll be able to eventually get to grips with the basic understanding of how to get by, there are a few Excel tips that not everybody will even know about let alone know how to d…
This Micro Tutorial will demonstrate on a Mac how to change the sort order for chart legend values and decrpyt the intimidating chart menu.
This Micro Tutorial will demonstrate how to use a scrolling table in Microsoft Excel using the INDEX function.

777 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question