Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people, just like you, are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
Solved

In memory database, shared memory, etc

Posted on 2004-09-13
7
264 Views
Last Modified: 2010-04-17
We have an application that is real-time and must update every second.   Sadly, we are using MS Access as are database.

The issue is the front end is handling drawing all the information on the screen which it gets from the database, and we have another service that uses a socket to retrieve info from a unix system and put this info into the MS Access DB.

I'm really worried about speed and having the socket program write all the time, it failing etc.

Isn't there a way i can do shared memory between applications in some sort of DLL?   Or is there a better mechanism between sharing high speed data between two apps (one running vb 6.0 right now, soon to be C#), other application is C#.

I'm just a little paranoid about a real-time application having two programs trying to retrieve info from a  Database every second!

Thanks


0
Comment
Question by:rkneal
7 Comments
 
LVL 11

Expert Comment

by:cjjclifford
ID: 12045106
have a look at Berkley DB, which supposedly supplies fast (optionally) in RAM databases... URL : http://www.sleepycat.com/

Btw, you could always replace the Access database with a strong opensource DB (PostgreSQL might be a good choice)...
0
 
LVL 3

Expert Comment

by:barryfandango
ID: 12046233
One or two database accesses per second shouldn't be too much of a performance problem.  Perhaps your newly designed application could use a messaging strategy instead of a polling strategy - that is, when something changes, a signal is sent to your GUI which then updates its display.

If you're looking for a small, high-performance database I would personally recommend not to try PGSQL and take a look at something like firebird (http://sourceforge.net/projects/firebird).  Postgre has great features but performance is not one of them.
0
 

Author Comment

by:rkneal
ID: 12046268
can you send an external event to a program from another program???

Lets say i'm in C# code that is doing the interface, could my vb 6.0 program send an event to it? Or are you saying we might want to look at something like a socket program between the two
0
Free Tool: SSL Checker

Scans your site and returns information about your SSL implementation and certificate. Helpful for debugging and validating your SSL configuration.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

 
LVL 3

Expert Comment

by:barryfandango
ID: 12046582
A socket would certainly be one way to do it.  Your C# program could listen on port XXX, and the vb program could send messages to let it know that the data has changed.  This has the added "bonus" of letting you run the processes on separate machines or across the globe if you ever needed to, but that's probably not desirable anyways for a real-time application.
0
 
LVL 3

Expert Comment

by:KeithWatson
ID: 12047848
If you must avoid the overhead of TCP/IP based communication, then it is possible to have windows programs communicate via shared memory using the Win32 API. Look up CreateFileMapping, MapViewOfFile and OpenFileMapping. The process that creates the shared memory would use CreateFileMapping and the process wishing to use the shared memory would use OpenFileMapping.

Hope this helps.
0
 
LVL 3

Accepted Solution

by:
Validor earned 250 total points
ID: 12049073
How much information is being moved from the Unix database to MSAccess?  You should only move as little information as possible, and move it mostly when other things are idle, if possible.  If the Unix source is fast enough, consider writing a web service to access the Unix source directly, and fetch only the necessary info for the task at hand.  If there is no way around the Unix -> Access shuffle that you're doing now, I would definitely suggest MSDE, Interbase, Firebird or MySQL.  Any of them would be faster than Access.  MSDE supports bulk-inserts which will handle HUGE inserts from the Unix source (sometimes 20,000 rows per second), but inserts ONLY- not updates.  MySQL is generally one of the fastest databases around (and is fairly cheap), but doesn't support all the features that the others do.  Interbase (v6.0) and Firebird are open-source and are embeddable.

There are several ways that you can share data between applications.  Sockets are one of the fastest for transmitting data and there are a host of libraries to help make the job easy.  C# comes with some swell objects in the FCL right off the bat.  I would also recommend the signalling methodology mentioned by barryfandango.  It will make updates faster and minimize traffic.  For simple notifications, Mailslots are easier than sockets, but are one-way and not guaranteed to be delivered (though I've NEVER seen them fail).

Memory-mapped files are the method mentioned by KeithWatson.  it works, but requires the data to already be there and the programs STILL have to communicate which parts are of this huge data store new and which parts are not, or they have to reload the whole data store every time they update.  It serves the same purpose as two applications using a database, which is usually better anyway.

My suggestion is to use MSDE with bulk-inserts to absorb updates from the Unix system, or a webservice to query the UNIX system directly.  If there is not much data coming in from the Unix system (even if it is frequntly), this should not be a problem.  I would still recommend another database over MSAccess, though.
0
 

Author Comment

by:rkneal
ID: 12049117
well, it is not a lot of data, but often. It's only basically 138 rows, need to read/write from them them every second.

I just hate MS Access, it's such a dog.  We have some experience at MySQL, so will try that or that berkely DB that can do RAM database.

Thanks
0

Featured Post

Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Go is an acronym of golang, is a programming language developed Google in 2007. Go is a new language that is mostly in the C family, with significant input from Pascal/Modula/Oberon family. Hence Go arisen as low-level language with fast compilation…
A short article about a problem I had getting the GPS LocationListener working.
An introduction to basic programming syntax in Java by creating a simple program. Viewers can follow the tutorial as they create their first class in Java. Definitions and explanations about each element are given to help prepare viewers for future …

829 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question