• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 479
  • Last Modified:

What is the most efficient language/way to test simple yet large amount of combinations?

Hi experts,

I have uploaded a spreadsheet to Google Document. You can see the file directly by clicking the link to google Doc below.
https://docs.google.com/spreadsheet/ccc?key=0AlhkkU7CRHI0dExrT2lOVHNSbTBTWURBY292cW5seWc

I am testing out 52,521,875 combinations (or more, depending on different survey results. With this one, it's 52,521,875 combinations), with the 1st combination being:
Sheet1 - Column A (54 names from A2~A55)
Sheet2 - Column A (54 names from A2~A55)
Sheet3 - Column A (54 names from A2~A55)
Sheet4 - Column A (54 names from A2~A55)
Sheet5 - Column A (54 names from A2~A55)

and 2nd combination being:
Sheet1 - Column A (54 names from A2~A55)
Sheet2 - Column A (54 names from A2~A55)
Sheet3 - Column A (54 names from A2~A55)
Sheet4 - Column A (54 names from A2~A55)
Sheet5 - Column B (54 names from B2~B55)

and 3rd combination being:
Sheet1 - Column A (54 names from A2~A55)
Sheet2 - Column A (54 names from A2~A55)
Sheet3 - Column A (54 names from A2~A55)
Sheet4 - Column A (54 names from A2~A55)
Sheet5 - Column C (54 names from B2~B55)


and so on until the last combination:

Sheet1 - Column AI (54 names from AI2~AI55)
Sheet2 - Column AI (54 names from AI2~AI55)
Sheet3 - Column AI (54 names from AI2~AI55)
Sheet4 - Column AI (54 names from AI2~AI55)
Sheet5 - Column AI (54 names from AI2~AI55)

For each combination, I am looking for CARMAKES that have ZERO occurance and output them on a separate Sheet. I can also choose to ONLY OUTPUT when there are MORE than X amount of CARMAKES with ZERO occurances.
(Eg. if X = 3, then Combination 1: if only 2 carmakes, toyota honda with zero occurance, then don't output.
If combination2: 3 carmakes, ford, bmw, fiat, with zero occurance, then output them on a separate sheet)


This is pretty much what I am looking to do.
However, given that there are 52,521,875 combination (and potentially more, with different amount of data input). I was advised that I was reaching Excel's limit in terms of the memory Excel allows and the limit of Excel's calculation speed.

Hence, I am wondering, what is the fastest/efficient way(language/program) to perform a test like this?

I am running a 2-year old Intel Q9300 CPU with 8GB of ram.

thank you very much for your help.
0
WiilingToPaypalAGoodCoder
Asked:
WiilingToPaypalAGoodCoder
  • 2
  • 2
1 Solution
 
TommySzalapskiCommented:
C or C++ would do the trick.
Since you have a quad core, I would just make 4 programs and have each one test 1/4 of the combinations then run all 4 at the same time.
You could learn how to make the program actually multi-threaded, but unless you plan to use multiple threads in the future, I wouldn't bother.
0
 
WiilingToPaypalAGoodCoderAuthor Commented:
Hi Tommy,

Thanks for the quick reply.
So with C or C++, they would be able to overcome
1) The memory limit that excel has for loading large-array?
2) The "relatively slower" speed of calculating 52,521,875 (or potentially more, depending on the different survey results) from excel?
0
 
TommySzalapskiCommented:
Yes. 52,521,875 combinations will be no problem. You could have it output something every 1 million combinations or so just so you know how it's going.

If I understand the problem correctly, you don't need a huge array, just a lot of tests on the array. If you do need large arrays, you will probably need to declare them like
int* array;
array = new int[52521875];

not
int array[52521875];

since the second method uses the stack and has stricter size limits on most systems.
0
 
WiilingToPaypalAGoodCoderAuthor Commented:
Thank you, Tommy, for your informative help.

I have msged you through "hire me" regarding application Design.
Thanks
0

Featured Post

Efficient way to get backups off site to Azure

This user guide provides instructions on how to deploy and configure both a StoneFly Scale Out NAS Enterprise Cloud Drive virtual machine and Veeam Cloud Connect in the Microsoft Azure Cloud.

  • 2
  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now