What is better to use a text file or sql db for data intensive app?
Posted on 2009-05-06
I'm building an application that will allow the user to import and name a hand full of data files (Excel data) each with a max of approx 10000 records, then the application will:
- compare record keys across the imported data files and report out what is missing between each data file and what is extra
- do some summing, division, max, mins, averages across all records in all tables
- remove and add several records across all data files depending on what's missing or extra across all data files
- rebuild all of the imported data files into a one file output
The user will import all of this data, run the application and take the output, then will do another project the same way, and then another and another each with different data and data files...sort of like creating Excel spreadsheets for various projects...each file is a totally different project, not the re-use of the same data or data structure
My question is: What is the best type of flat DB to use in terms of speed for user, ease to build the application? : text files, sql db, other?
Thank you very much!