Solved

Best way to share data

Posted on 2004-10-12
18
168 Views
Last Modified: 2010-04-05
I have a server application that gathers data from many different data sources, and stored them in several local arrays. This data is then used to calculate live statistics for live displays.

I need to extend this server application to allow for custom modules to be written that will use this data for custom applications for our customers (In my specific case right now, I need to make a telnet server that will spit out data in a delimited format)

Since I have to modify the server anyway we would like to make it easier to add extensions like this one in the future.

Finally, My question is: How can I make it so that I can read (read only is fine) the data from an external application or a DLL that I load? I'm sure I'll have to use memory mapped files some how, but how can I accomplish this without having to change the structure of the existing server application too much?
0
Comment
Question by:rbohac
  • 8
  • 6
  • 2
  • +1
18 Comments
 
LVL 17

Accepted Solution

by:
Wim ten Brink earned 200 total points
ID: 12287108
There are just too many techniques to share data between processes. Memory mapped files is just one method. Through TCP/IP you could change your application in a server that accepts requests for information from other processes that can even be on other systems.
The use of COM would also make it possible, by creating a global singleton object. Thus, all applications that want to get your data get a reference to the same COM object. Once the last application is done with it, the COM object would just clean itself up again.
Named pipes are also a possibility, though.

However, the easiest way, with the least impact is by adding TCP/IP support to your application. Add a TidTCPServer component to your project from the Indy components and let it accept incoming connections. Then all other applications just have to connect to your server applications, make requests and wait for the response. It's not too difficult, once you're familiar with it.
The biggest advantages is that it has the smallest learning curve and is a lot less buggy than using mapped files.
0
 
LVL 6

Author Comment

by:rbohac
ID: 12289142
I wish I could use a TCP socket, but that is way too much data to have to copy. Somehow I'm going to need direct memory access.

I'm not familiar with COM objects at all, would that be a preferred method over memory mapped files?

Also, I don't want to have to do the data formatting from within the server application. Basically I need to set it up so that I can access the data from the other applications without having to change the server (or restart it) at all
0
 
LVL 6

Author Comment

by:rbohac
ID: 12289355
What about this idea:

Instead of direct memory access, could I use sendmessage combined with HWND_BROADCAST to broadcast the individual records that were changed to all applications, which would then allow the other applications to handle their own data collection? Can WM_COPYDATA be broadcasted?
0
 
LVL 12

Assisted Solution

by:Lee_Nover
Lee_Nover earned 200 total points
ID: 12291353
I'd go with MMF + Events + threads
create a MMF with the required security descriptors, do the same with events
then a background thread would wait for this event to pulse
everytime someone changes the data it sets the event and releases the other waiting threads
use two events for multi read exclusive write protection
you can check my global var implementation (also includes unfinished waiting threads using the mentioned event scheme)
http://leenover.homeip.net/isapi/pas2html.dll/pas2html?File=/delphi/lnGlobalVar
0
 

Assisted Solution

by:joncmora
joncmora earned 100 total points
ID: 12296269
If all apps will be running on a single machine, you'll be coding a lot less if you use simple custom windows messages. Passing the address of your arrays in wParam and/or lParam.
0
 
LVL 12

Expert Comment

by:Lee_Nover
ID: 12296309
joncmora: and do what with those addresses ? they're useless in other processes .. that's where MMF's come in to play :)
0
 

Expert Comment

by:joncmora
ID: 12296405
>>and do what with those addresses ? they're useless in other processes

Lee_Nover: haven't done this before? :)
0
 
LVL 6

Author Comment

by:rbohac
ID: 12297413
I'd like to stay away from MMF if I can just because it would take a total restructuring of how the data is stored in the server application. It has been stable for quite some time, and I don't want to risk that.

joncmora, As far as I know, you can't pass pointers to data structures without using WM_COPYDATA.

What about sharemem or fastsharemem? Could I somehow just create DLL's that have access to the programs memory? I guess I could just read and load them from a "plugins" directory somehow.
0
Highfive + Dolby Voice = No More Audio Complaints!

Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

 
LVL 6

Author Comment

by:rbohac
ID: 12319129
I think I'm just going to end up using dlls and fastsharemem
0
 
LVL 12

Expert Comment

by:Lee_Nover
ID: 12319530
any data in the dll is per process
I guess you can broadcast the wm_copydata message
0
 
LVL 6

Author Comment

by:rbohac
ID: 12319553
I'm just going to have a standard list of exports, and if the DLL contains that export, I will pass the data to it
0
 
LVL 12

Expert Comment

by:Lee_Nover
ID: 12319767
and just how will one application get the server's data ? dll proxying won't work
0
 
LVL 6

Author Comment

by:rbohac
ID: 12319881
Unless theres something I'm missing, it seems to be working just fine for me. Isn't that the whole point of sharemem/fastsharemem?

Or seriously, am I missing somehting here?
0
 
LVL 12

Expert Comment

by:Lee_Nover
ID: 12320029
errr .. I must be missing something here :)
can you give a brief explanation how you implemented this ?
0
 
LVL 6

Author Comment

by:rbohac
ID: 12320107
The server process grabs *.dll from a "plugins" directory
I use loadlibrary on all of them, and scan for a preset list of exports by using GetProcAddess. If it returns a valid handle, that that dll uses that function.

From then on, I pass each dll events using the functions/procedures retrieved via getprocaddress.

Using this method, I do have to pass the data along with it, but it keeps me from having to restructure the entire server application to be able to use MMF
0
 
LVL 12

Expert Comment

by:Lee_Nover
ID: 12320269
and the client apps also load those dlls ? .. ah never mind .. can't think clearly now and don't want to waste your time :)
glad you got it working :P
0
 
LVL 6

Author Comment

by:rbohac
ID: 12320328
Client apps have nothing to do with this. They do connect to the server, but this is totally seperate
0

Featured Post

Maximize Your Threat Intelligence Reporting

Reporting is one of the most important and least talked about aspects of a world-class threat intelligence program. Here’s how to do it right.

Join & Write a Comment

Have you ever had your Delphi form/application just hanging while waiting for data to load? This is the article to read if you want to learn some things about adding threads for data loading in the background. First, I'll setup a general applica…
Introduction Raise your hands if you were as upset with FireMonkey as I was when I discovered that there was no TListview.  I use TListView in almost all of my applications I've written, and I was not going to compromise by resorting to TStringGrid…
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.
In this tutorial you'll learn about bandwidth monitoring with flows and packet sniffing with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're interested in additional methods for monitoring bandwidt…

705 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now