Which Database shouls I use for fast data access with Visual C++?

Hi,
I am writing a C++ program that reads in Forex Data of Currencies values.
Reading it to the memory from text files takes a long time.

I am using a PC with vista 64 bit and Visual C++ 8GB Ram.
1) Which database should I use?
2) Any speed reference comparing to memory access?

3) Which Databases can use a lot of Ram for faster results?
 e.g. 4 GB of Data that is hold in Ram?

Any article abou it?
Thanks

star90Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Kent OlsenData Warehouse Architect / DBACommented:
Hi Star,

All of the modern RDBMS will use as much or little memory as you can configure.  If your server has 32 or even 64GB, you can use it.

The real question is I/O volume.  How many transactions/second do you need to sustain?  How many in burst?

MySQL is probably sufficient for your needs and is free.  If you work for someone that wants a traditional vendor and support, I'd recommend DB2.  Of course, you'll get just as many people endorsing Oracle, which is just as capable.


Kent
0
enachemcCommented:
Oracle offers a free version, but will limit it's use of resources.
0
star90Author Commented:
I have one PC so I want the Database and the client to be on the same Computer.
If for example I want to be able to run on 8 million values of a specific currency for a segment.
Do I need first to read the Data from the Database first into the memory ?
Or do I access it value by value?

Assuming I want to read 8 million of floats values, how long would it take?

I also want to understand, Assuming I keep some data in my program using  STL containers, how do I convert it to Databases? Is it complicated?
Thanks



0
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

Kent OlsenData Warehouse Architect / DBACommented:
Hi Star,

8,000,000 rows is pretty modest with today's processors.  Of course, reading all 8 million for a single query will take a bit of time.

But that's the job of the RDBMS.  Managing the data and controlling the I/O.

Depending on the data, structure, and usage, you may be able to make that very, very efficient.  But it's not a "here's the solution" kind of venture.  It takes a knowledge of your data, and a good database design to make it work really well.


Kent
0
FerrostiCommented:
One thing you should keep in mind is selecting ALL values from RDBMS is no way faster than reading a file at once unless you´ve got in in RAM.

Please correct me if I am wrong, but I think there is a lack of understanding of client/server and what a RDBMS does?
One won´t receive all data with his client and work on it locally when using a RDBMS. One would send a SQL statement to it, select its data there and only work on a small resultset on the client. This is very important to consider!

mySQL as well as postgres and other non-free won´t have any trouble handling this. This would just be a matter of your personal taste. E.g. I´d go for mySQL since I am not familiar with postgres and that´s the only reason.
0
star90Author Commented:
Where do I start?
I understand that Visual c++ 2010 has SQlite inside.
Is it a good starting point?
any good tutorial about it?

Assuming some of my data is stored inSTL  vectors and STL maps, how do I save it to the database?
0
Kent OlsenData Warehouse Architect / DBACommented:
Hi Star,

Designing the application and the database are related, but only loosely so.  A good database design is critical, though it should be tolerant of the application's (client's) needs.

STL vectors and maps have to equivalent in SQLite.  You'll have to engineer the database to accommodate the objects in the vectors, and explicitly transfer data to/from the vectors and database.  Depending on your needs, you might find that you want to drop the vectors from the application and let the RDBMS manage all of that.


Kent
0
star90Author Commented:
How do I start practically with Visual c++ and SQlite?
DO I need to install SQL server?
Any good tutorial on how to start?
0
Kent OlsenData Warehouse Architect / DBACommented:

Hi star,

SQL Server and SQLite are two entirely different things.  They are NOT easily compatible.

SQLite has extensive documentation, but it's not particularly user friendly.  It's more reference material than tutorial.

Still, here it is:

  http://www.sqlite.org/


Kent
0
star90Author Commented:
Assuming I can use SQlite or SQL Server (Which comes with VC++ 2010.
Which one is faster? which one is easier to use and study?
0
Kent OlsenData Warehouse Architect / DBACommented:
I don't believe that you'll see much speed difference between the two.

SQL Server is probably easier to use, but I'm pretty sure that VC++ does NOT grant a license to run SQL Server.  You can develop clients against a SQL Server database, but you need a separate license to run the server (which is, ultimately, the database).


0
FerrostiCommented:
No, MS SQL Server is always an extra and there is no product which ships including MS SQL Server.
Since you only have a PC for testing I assume you have got a client OS like Windows 7 or XP running on it? SQL Server requires a server OS like Windows 2008 R2 Server for installation.

Saying this I´d probably go for postgres in your case.
0
star90Author Commented:
Thanks.
Assuming I choose the Microsoft SQL server.
Any good starting point with examples?
0
star90Author Commented:
Hi Ferrosti,
Thanks for the information I did not know that I can´t run the SQL server on the same machine.
I will read about postgresql.
0
Kent OlsenData Warehouse Architect / DBACommented:
Hi Star,

If you need to run a separate server (so that multiple clients can access the database) you might check MySQL, too.  Depending on where you are in the world you'll probably find more people that are familiar with MySQL than PostGRES.

If you are building everything into a single program that doesn't need multiple clients, SQLite will do just fine.


Kent
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Databases

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.