Which Sorting Algorithm to use for large datasets ?
Posted on 2008-10-07
I am writing a C program to read a comma separated csv file ( which will have anywhere between 300k to 1.5 million rows). At the end of the program I will be outputting a csv file which will have double the number of input rows(I also sort it based on a DateTime field in the file ).
1) I am using Insertion Sort algorithm where in I sort as I create the nodes. This takes around 1.5 hrs for 500k rows. But insertion sort isnt going to work for >1 million rows. It's going to take a really long time!!
2) Will it be any better If I QuickSort ? I know it will be faster but if i use Quicksort, I will have to first create nodes(eg;say for a million) and then again do a sort.
3)Or is C a bad choice for this kind of a problem ?