fastest or most efficient way to store lots of data

jscharpf
jscharpf used Ask the Experts™
on
I have a VB program that will be generating alot of raw data.
For example.. it will be running a battery test, and I will be logging about 100 single precision values 4 times per second for several days...
I started by just adding records to and Access database, but it appears that could get very large.. and maybe slow down??

Is using a text file faster than Access? Does it take more or less memory to store the same amount of data?

Thanks

jeff
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
I recommend with that kind of data, if you don't need to access the data in real-time, that you gather the data in local memory until you reach a threshold, then pass that data off to be saved.

With your 400 Single entries per second, you may want to write out (asynchronously) the 24000 entries to a DB every minute.  With memory being cheap and the assumption that the box will probably not be in use during these tests by someone, this is a workable solution.  I recommend a DB for the full-features you will want to evaluate these numbers.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial