Solved

Which Database shouls I use for fast data access with Visual C++?

Posted on 2011-02-14
15
488 Views
Last Modified: 2012-05-11
Hi,
I am writing a C++ program that reads in Forex Data of Currencies values.
Reading it to the memory from text files takes a long time.

I am using a PC with vista 64 bit and Visual C++ 8GB Ram.
1) Which database should I use?
2) Any speed reference comparing to memory access?

3) Which Databases can use a lot of Ram for faster results?
 e.g. 4 GB of Data that is hold in Ram?

Any article abou it?
Thanks

0
Comment
Question by:star90
  • 6
  • 6
  • 2
  • +1
15 Comments
 
LVL 45

Expert Comment

by:Kdo
ID: 34887794
Hi Star,

All of the modern RDBMS will use as much or little memory as you can configure.  If your server has 32 or even 64GB, you can use it.

The real question is I/O volume.  How many transactions/second do you need to sustain?  How many in burst?

MySQL is probably sufficient for your needs and is free.  If you work for someone that wants a traditional vendor and support, I'd recommend DB2.  Of course, you'll get just as many people endorsing Oracle, which is just as capable.


Kent
0
 
LVL 12

Expert Comment

by:enachemc
ID: 34887815
Oracle offers a free version, but will limit it's use of resources.
0
 

Author Comment

by:star90
ID: 34887948
I have one PC so I want the Database and the client to be on the same Computer.
If for example I want to be able to run on 8 million values of a specific currency for a segment.
Do I need first to read the Data from the Database first into the memory ?
Or do I access it value by value?

Assuming I want to read 8 million of floats values, how long would it take?

I also want to understand, Assuming I keep some data in my program using  STL containers, how do I convert it to Databases? Is it complicated?
Thanks



0
 
LVL 45

Expert Comment

by:Kdo
ID: 34887965
Hi Star,

8,000,000 rows is pretty modest with today's processors.  Of course, reading all 8 million for a single query will take a bit of time.

But that's the job of the RDBMS.  Managing the data and controlling the I/O.

Depending on the data, structure, and usage, you may be able to make that very, very efficient.  But it's not a "here's the solution" kind of venture.  It takes a knowledge of your data, and a good database design to make it work really well.


Kent
0
 
LVL 6

Expert Comment

by:Ferrosti
ID: 34888119
One thing you should keep in mind is selecting ALL values from RDBMS is no way faster than reading a file at once unless you´ve got in in RAM.

Please correct me if I am wrong, but I think there is a lack of understanding of client/server and what a RDBMS does?
One won´t receive all data with his client and work on it locally when using a RDBMS. One would send a SQL statement to it, select its data there and only work on a small resultset on the client. This is very important to consider!

mySQL as well as postgres and other non-free won´t have any trouble handling this. This would just be a matter of your personal taste. E.g. I´d go for mySQL since I am not familiar with postgres and that´s the only reason.
0
 

Author Comment

by:star90
ID: 34888121
Where do I start?
I understand that Visual c++ 2010 has SQlite inside.
Is it a good starting point?
any good tutorial about it?

Assuming some of my data is stored inSTL  vectors and STL maps, how do I save it to the database?
0
 
LVL 45

Expert Comment

by:Kdo
ID: 34888180
Hi Star,

Designing the application and the database are related, but only loosely so.  A good database design is critical, though it should be tolerant of the application's (client's) needs.

STL vectors and maps have to equivalent in SQLite.  You'll have to engineer the database to accommodate the objects in the vectors, and explicitly transfer data to/from the vectors and database.  Depending on your needs, you might find that you want to drop the vectors from the application and let the RDBMS manage all of that.


Kent
0
Highfive + Dolby Voice = No More Audio Complaints!

Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

 

Author Comment

by:star90
ID: 34888521
How do I start practically with Visual c++ and SQlite?
DO I need to install SQL server?
Any good tutorial on how to start?
0
 
LVL 45

Expert Comment

by:Kdo
ID: 34888545

Hi star,

SQL Server and SQLite are two entirely different things.  They are NOT easily compatible.

SQLite has extensive documentation, but it's not particularly user friendly.  It's more reference material than tutorial.

Still, here it is:

  http://www.sqlite.org/


Kent
0
 

Author Comment

by:star90
ID: 34888723
Assuming I can use SQlite or SQL Server (Which comes with VC++ 2010.
Which one is faster? which one is easier to use and study?
0
 
LVL 45

Expert Comment

by:Kdo
ID: 34888749
I don't believe that you'll see much speed difference between the two.

SQL Server is probably easier to use, but I'm pretty sure that VC++ does NOT grant a license to run SQL Server.  You can develop clients against a SQL Server database, but you need a separate license to run the server (which is, ultimately, the database).


0
 
LVL 6

Assisted Solution

by:Ferrosti
Ferrosti earned 100 total points
ID: 34888862
No, MS SQL Server is always an extra and there is no product which ships including MS SQL Server.
Since you only have a PC for testing I assume you have got a client OS like Windows 7 or XP running on it? SQL Server requires a server OS like Windows 2008 R2 Server for installation.

Saying this I´d probably go for postgres in your case.
0
 

Author Comment

by:star90
ID: 34888942
Thanks.
Assuming I choose the Microsoft SQL server.
Any good starting point with examples?
0
 

Author Comment

by:star90
ID: 34889033
Hi Ferrosti,
Thanks for the information I did not know that I can´t run the SQL server on the same machine.
I will read about postgresql.
0
 
LVL 45

Accepted Solution

by:
Kdo earned 200 total points
ID: 34889083
Hi Star,

If you need to run a separate server (so that multiple clients can access the database) you might check MySQL, too.  Depending on where you are in the world you'll probably find more people that are familiar with MySQL than PostGRES.

If you are building everything into a single program that doesn't need multiple clients, SQLite will do just fine.


Kent
0

Featured Post

What Should I Do With This Threat Intelligence?

Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

Join & Write a Comment

Suggested Solutions

Entity Framework is a powerful tool to help you interact with the DataBase but still doesn't help much when we have a Stored Procedure that returns more than one resultset. The solution takes some of out-of-the-box thinking; read on!
Read about achieving the basic levels of HRIS security in the workplace.
Video by: Steve
Using examples as well as descriptions, step through each of the common simple join types, explaining differences in syntax, differences in expected outputs and showing how the queries run along with the actual outputs based upon a simple set of dem…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now