Solved

What is better to use a text file or sql db for data intensive app?

Posted on 2009-05-06
15
192 Views
Last Modified: 2012-05-06
I'm building an application that will allow the user to import and name a hand full of data files (Excel data) each with a max of approx 10000 records, then the application will:
- compare record keys across the imported data files and report out what is missing between each data file and what is extra
- do some summing, division, max, mins, averages across all records in all tables
- remove and add several records across all data files depending on what's missing or extra across all data files
- rebuild all of the imported data files into a one file output

The user will import all of this data, run the application and take the output, then will do another project the same way, and then another and another each with different data and data files...sort of like creating Excel spreadsheets for various projects...each file is a totally different project, not the re-use of the same data or data structure

My question is: What is the best type of flat DB to use in terms of speed for user, ease to build the application? : text files, sql db, other?

Thank you very much!
0
Comment
Question by:adamchicago
  • 4
  • 4
  • 2
  • +2
15 Comments
 
LVL 60

Expert Comment

by:chapmandew
ID: 24318816
sql database...FOR SURE.  you can use sql express...its free!
0
 
LVL 60

Expert Comment

by:chapmandew
ID: 24318825
0
 
LVL 33

Expert Comment

by:jppinto
ID: 24319017
I agree with chapmandew...text file only to use for keeping very small pieces of data. For structured data, use an database engine like SQL Express. As said, it's free and ideal for small applications.

jppinto
0
 
LVL 1

Author Comment

by:adamchicago
ID: 24319084
If the user begins a project which generates a SQL DB, can they have complete freedom to save the SQL DB anywhere on their computer and then possibly copy and paste it to a new location and reopen as easily as a text file or as easily as an MS Office app such as Excel, PowerPt etc?
0
 
LVL 84
ID: 24319217
Not with SQL Server. Of course, why would you need to do this with a relational database?

You can do this with an Access database ...
0
 
LVL 49

Expert Comment

by:Gustav Brock
ID: 24319260
No, doable but not easily.

For 6 times 10000 records Access/JET will definitely be the fastest datastore.
And this single mdb file can be copied and moved just like any text file.

/gustav
0
 
LVL 33

Expert Comment

by:jppinto
ID: 24319271
For what you want to do...Access is the best option for you. It's fully integrated with other Office tools.
0
Enterprise Mobility and BYOD For Dummies

Like “For Dummies” books, you can read this in whatever order you choose and learn about mobility and BYOD; and how to put a competitive mobile infrastructure in place. Developed for SMBs and large enterprises alike, you will find helpful use cases, planning, and implementation.

 
LVL 1

Author Comment

by:adamchicago
ID: 24319348
I want to stay away from a paid licensed DB provider such as Access because not all users of the this app will have an Access license.  From what I understand if the app uses a SQL DB, anyone who downloads SQL Server Express for free from the MS website will be able to use this app without having to purchase any kind of DB license...am I wrong?
0
 
LVL 49

Assisted Solution

by:Gustav Brock
Gustav Brock earned 100 total points
ID: 24319382
An mdb file is free to create and use.

/gustav
0
 
LVL 84

Assisted Solution

by:Scott McDaniel (Microsoft Access MVP - EE MVE )
Scott McDaniel (Microsoft Access MVP - EE MVE ) earned 100 total points
ID: 24319548
the only time you need a license for Access is if you use the Access interface. In your case, you're actually using a Jet database, which is freely distributable, assuming you own one of the MS development environments (like VB, Access, etc).
0
 
LVL 1

Author Comment

by:adamchicago
ID: 24320277
So what is the advantage of using a MDB (Access like DB) versus a MDF (SQL DB)?   Which one offers more file portability for the app user (VB.Net app I'm building) and which one has more robust analytics (VB.Net app will process data faster and is easier for me to build)?

Thanks again...I've already sunk a number of hours into this app and just want to make sure I'm on the right  path;)
0
 
LVL 60

Accepted Solution

by:
chapmandew earned 300 total points
ID: 24321654
SQL Express (mdf) is for sure your best option here.  It is more robust, has a better processing engine, more security features, and allows for more concurrent users.
0
 
LVL 1

Author Closing Comment

by:adamchicago
ID: 31578687
Thanks for your help!
0
 
LVL 60

Expert Comment

by:chapmandew
ID: 24328682
These were not B answers....
0

Featured Post

[Webinar] Disaster Recovery and Cloud Management

Learn from Unigma and CloudBerry industry veterans which providers are best for certain use cases and how to lower cloud costs, how to grow your Managed Services practice in IaaS clouds, and how to utilize public cloud for Disaster Recovery

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

This article explains how to reset the password of the sa account on a Microsoft SQL Server.  The steps in this article work in SQL 2005, 2008, 2008 R2, 2012, 2014 and 2016.
This article shows gives you an overview on SQL Server 2016 row level security. You will also get to know the usages of row-level-security and how it works
In Microsoft Access, learn how to “cascade” or have the displayed data of one combo control depend upon what’s entered in another. Base the dependent combo on a query for its row source: Add a reference to the first combo on the form as criteria i…
Viewers will learn how to use the SELECT statement in SQL and will be exposed to the many uses the SELECT statement has.

867 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now