Solved

Which Database shouls I use for fast data access with Visual C++?

Posted on 2011-02-14
15
499 Views
Last Modified: 2012-05-11
Hi,
I am writing a C++ program that reads in Forex Data of Currencies values.
Reading it to the memory from text files takes a long time.

I am using a PC with vista 64 bit and Visual C++ 8GB Ram.
1) Which database should I use?
2) Any speed reference comparing to memory access?

3) Which Databases can use a lot of Ram for faster results?
 e.g. 4 GB of Data that is hold in Ram?

Any article abou it?
Thanks

0
Comment
Question by:star90
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 6
  • 2
  • +1
15 Comments
 
LVL 45

Expert Comment

by:Kent Olsen
ID: 34887794
Hi Star,

All of the modern RDBMS will use as much or little memory as you can configure.  If your server has 32 or even 64GB, you can use it.

The real question is I/O volume.  How many transactions/second do you need to sustain?  How many in burst?

MySQL is probably sufficient for your needs and is free.  If you work for someone that wants a traditional vendor and support, I'd recommend DB2.  Of course, you'll get just as many people endorsing Oracle, which is just as capable.


Kent
0
 
LVL 12

Expert Comment

by:enachemc
ID: 34887815
Oracle offers a free version, but will limit it's use of resources.
0
 

Author Comment

by:star90
ID: 34887948
I have one PC so I want the Database and the client to be on the same Computer.
If for example I want to be able to run on 8 million values of a specific currency for a segment.
Do I need first to read the Data from the Database first into the memory ?
Or do I access it value by value?

Assuming I want to read 8 million of floats values, how long would it take?

I also want to understand, Assuming I keep some data in my program using  STL containers, how do I convert it to Databases? Is it complicated?
Thanks



0
The Eight Noble Truths of Backup and Recovery

How can IT departments tackle the challenges of a Big Data world? This white paper provides a roadmap to success and helps companies ensure that all their data is safe and secure, no matter if it resides on-premise with physical or virtual machines or in the cloud.

 
LVL 45

Expert Comment

by:Kent Olsen
ID: 34887965
Hi Star,

8,000,000 rows is pretty modest with today's processors.  Of course, reading all 8 million for a single query will take a bit of time.

But that's the job of the RDBMS.  Managing the data and controlling the I/O.

Depending on the data, structure, and usage, you may be able to make that very, very efficient.  But it's not a "here's the solution" kind of venture.  It takes a knowledge of your data, and a good database design to make it work really well.


Kent
0
 
LVL 6

Expert Comment

by:Ferrosti
ID: 34888119
One thing you should keep in mind is selecting ALL values from RDBMS is no way faster than reading a file at once unless you´ve got in in RAM.

Please correct me if I am wrong, but I think there is a lack of understanding of client/server and what a RDBMS does?
One won´t receive all data with his client and work on it locally when using a RDBMS. One would send a SQL statement to it, select its data there and only work on a small resultset on the client. This is very important to consider!

mySQL as well as postgres and other non-free won´t have any trouble handling this. This would just be a matter of your personal taste. E.g. I´d go for mySQL since I am not familiar with postgres and that´s the only reason.
0
 

Author Comment

by:star90
ID: 34888121
Where do I start?
I understand that Visual c++ 2010 has SQlite inside.
Is it a good starting point?
any good tutorial about it?

Assuming some of my data is stored inSTL  vectors and STL maps, how do I save it to the database?
0
 
LVL 45

Expert Comment

by:Kent Olsen
ID: 34888180
Hi Star,

Designing the application and the database are related, but only loosely so.  A good database design is critical, though it should be tolerant of the application's (client's) needs.

STL vectors and maps have to equivalent in SQLite.  You'll have to engineer the database to accommodate the objects in the vectors, and explicitly transfer data to/from the vectors and database.  Depending on your needs, you might find that you want to drop the vectors from the application and let the RDBMS manage all of that.


Kent
0
 

Author Comment

by:star90
ID: 34888521
How do I start practically with Visual c++ and SQlite?
DO I need to install SQL server?
Any good tutorial on how to start?
0
 
LVL 45

Expert Comment

by:Kent Olsen
ID: 34888545

Hi star,

SQL Server and SQLite are two entirely different things.  They are NOT easily compatible.

SQLite has extensive documentation, but it's not particularly user friendly.  It's more reference material than tutorial.

Still, here it is:

  http://www.sqlite.org/


Kent
0
 

Author Comment

by:star90
ID: 34888723
Assuming I can use SQlite or SQL Server (Which comes with VC++ 2010.
Which one is faster? which one is easier to use and study?
0
 
LVL 45

Expert Comment

by:Kent Olsen
ID: 34888749
I don't believe that you'll see much speed difference between the two.

SQL Server is probably easier to use, but I'm pretty sure that VC++ does NOT grant a license to run SQL Server.  You can develop clients against a SQL Server database, but you need a separate license to run the server (which is, ultimately, the database).


0
 
LVL 6

Assisted Solution

by:Ferrosti
Ferrosti earned 100 total points
ID: 34888862
No, MS SQL Server is always an extra and there is no product which ships including MS SQL Server.
Since you only have a PC for testing I assume you have got a client OS like Windows 7 or XP running on it? SQL Server requires a server OS like Windows 2008 R2 Server for installation.

Saying this I´d probably go for postgres in your case.
0
 

Author Comment

by:star90
ID: 34888942
Thanks.
Assuming I choose the Microsoft SQL server.
Any good starting point with examples?
0
 

Author Comment

by:star90
ID: 34889033
Hi Ferrosti,
Thanks for the information I did not know that I can´t run the SQL server on the same machine.
I will read about postgresql.
0
 
LVL 45

Accepted Solution

by:
Kent Olsen earned 200 total points
ID: 34889083
Hi Star,

If you need to run a separate server (so that multiple clients can access the database) you might check MySQL, too.  Depending on where you are in the world you'll probably find more people that are familiar with MySQL than PostGRES.

If you are building everything into a single program that doesn't need multiple clients, SQLite will do just fine.


Kent
0

Featured Post

Three Reasons Why Backup is Strategic

Backup is strategic to your business because your data is strategic to your business. Without backup, your business will fail. This white paper explains why it is vital for you to design and immediately execute a backup strategy to protect 100 percent of your data.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

As technology users and professionals, we’re always learning. Our universal interest in advancing our knowledge of the trade is unmatched by most industries. It’s a curiosity that makes sense, given the climate of change. Within that, there lies a…
Recently, Microsoft released a best-practice guide for securing Active Directory. It's a whopping 300+ pages long. Those of us tasked with securing our company’s databases and systems would, ideally, have time to devote to learning the ins and outs…
Video by: Steve
Using examples as well as descriptions, step through each of the common simple join types, explaining differences in syntax, differences in expected outputs and showing how the queries run along with the actual outputs based upon a simple set of dem…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

751 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question