We help IT Professionals succeed at work.

How do I create a background data stream in a C++ DLL?

precords asked
I'm new to C++ and this has been quite the learning curve.  I've got a dll I'm creating to access streaming market data and perform operations on it and return results to tradestation so must use __stdcall it to be compatible.  Currently this is running in Visual Studio 6 with no MFC.  My problem is that I have to use a sleep command to keep the quotes coming (or the faucet gets turned off when the return happens), BUT then the dll is locked there forever and I can't access stuff, soooo I need the connection enabled by the Ewrapper and EclientL0 and subsequent REQMKTDATA (turns on data faucet) to be in a background thread so that I can still access and call things in the main thread.  How do I do this??  I want to turn it on in the dll when DLL_PROCESS_ATTACH and then exit the background thread when DLL_PROCESS_DETACH occurs.  Also wondering if the variables in the background thread are still available to the foreground thread when they are setup in the top of the main.cpp?
Many thanks in advance for your help!!!!!

Mr. Beginner
*************  Here's the code for the DLLMAIN
( HANDLE hModule, DWORD  ul_reason_for_call, LPVOID lpReserved )
    switch ( ul_reason_for_call )
            case DLL_PROCESS_ATTACH:{
                        fp=fopen("C:/tradingfiles/mdeltadlllog.txt", "a+");  //cj
                        fprintf(fp, "Main dll case process attach\n" );
                        fclose(fp);  //cj
                        //pNewMyconnection = new EC;
            case DLL_THREAD_ATTACH:{
                        fp=fopen("C:/tradingfiles/mdeltadlllog.txt", "a+");  //cj
                        fprintf(fp, "Main dll case thread attach\n" );
                        fclose(fp);  //cj
            case DLL_THREAD_DETACH:{
                        fp=fopen("C:/tradingfiles/mdeltadlllog.txt", "a+");  //cj
                        fprintf(fp, "Main dll case thread detach\n" );
                        fclose(fp);  //cj
            case DLL_PROCESS_DETACH:{
                        fp=fopen("C:/tradingfiles/mdeltadlllog.txt", "a+");  //cj
                        fprintf(fp, "Main dll case process detach\n" );
                        fclose(fp);  //cj

    return TRUE ;
int __stdcall MD_connect
    if ( iID >= 0 )  {
		MyEWrapper	EW;
		EClientL0*	EC = EClientL0::New( &EW );
		//printf( "ClientVersion = %d\n", EC->clientVersion() );
		IDint = iID; 
		IPstring = (const char*)sIP;
		mysym = (const char*)sSYMBOL;
		myexpiry = (const char*)sEXP;
		mysecType = (const char*)sSECTYPE;
		mycurrency = (const char*)sCURRENCY;
		myexchange = (const char*)sEXCHANGE;
		if( EC->eConnect( &IPstring[0], 7496, IDint ) )  
			{print stuff
		Contract C;
		C.symbol	=  &mysym[0];
		C.expiry	= &myexpiry[0];
		C.secType	= &mysecType[0];
		C.currency	= &mycurrency[0];
		C.exchange = &myexchange[0];
		EC->reqContractDetails( C );
		EC->reqMktData( 10, C, "", false );
		while( EC->IsConnected()) {	Sleep( 1000 ); }
		if (EC->IsConnected()) { return 1; } else {return -1;}
		return -2 ; // Error code:  Element location out of range

Open in new window

Watch Question

Senior Software Engineer (Avast)
Use CreateThread to create a new thread. You must pass this function ythe address of a function that becomes the code of execution for the new "worker thread". This function should contain the code that does the work until such time as the main thread signals it to stop.

The simplest way to signal it is to use a condition variable that is shared between the main thread and the worker thread (passing it is as the LPVOID lpParameter is the simplest way to share it). This is a variable (ideally based upon type sig_atomic_t as it is a data type guaranteed to be read/written atomically x-platform) that MUST be defined volatile otherwise the changes in one thread may not be seen in another (due to compiler optimization).

You can set the condition variable to, say, initially 0 and then you want the worker thread to terminate you set this to 1.  The worker thread, as soon as it sees this condition variable has been set to 1, should terminate. You can then use the WaitForSingleObject() function in the main thread to wait for the worker to terminate before the main thread exits.




Well I think I've solved the first part.  Now I'm having trouble disconnecting with endthread().  Won't seem to kick out gracefully.
evilrixSenior Software Engineer (Avast)
Please clarify your problem. To end your thread all you need to do is signal the worker function to terminate, It does this by just returning like any other function. This will implicitly end the thread.
Deepu AbrahamR & D Engineering Manager
See the Exit thread:

 VOID WINAPI ExitThread(
  __in  DWORD dwExitCode

ExitThread is the preferred method of exiting a thread in C code. However, in C++ code, the thread is exited before any destructors can be called or any other automatic cleanup can be performed. Therefore, in C++ code, you should return from your thread function.

When this function is called (either explicitly or by returning from a thread procedure), the current thread's stack is deallocated, all pending I/O initiated by the thread is canceled, and the thread terminates. The entry-point function of all attached dynamic-link libraries (DLLs) is invoked with a value indicating that the thread is detaching from the DLL.

If the thread is the last thread in the process when this function is called, the thread's process is also terminated.

The state of the thread object becomes signaled, releasing any other threads that had been waiting for the thread to terminate. The thread's termination status changes from STILL_ACTIVE to the value of the dwExitCode parameter.

Terminating a thread does not necessarily remove the thread object from the operating system. A thread object is deleted when the last handle to the thread is closed.
Deepu AbrahamR & D Engineering Manager
Have a look at this link as well.
Best Regards,
evilrixSenior Software Engineer (Avast)

>> See the Exit thread:

There is no need to use EndThread when the threadproc function can just return, which effectively terminates the thread.

Actually, if you are using the C/C++ runtime in your code you shouldn't even use CreateThread or EndThread at all; rather, you should use _beginthread and _endthread (except you don't need to if your threadproc function just returns). This is because the other two functions by-pass the C/C++ runtime Thread Local Storage allocation and clean-up mechanisms. If the TLS allocation is by-passed it is still implicitly created; however, it is never released and as such a memory leak occurs!


Multi-threading tutorial
>>>> to be in a background thread so that I can still access and call things in the main thread.

You better would invoke an additionally server process (.exe) or activate a service rather than doing all the stuff in the dll. The dll runs in the context of the calling process and it is not easy to keep the dll running beyond one call and it isn't trivial either to hold data so that it is shared between all callers. You better turn the dll to be only the client interface of a client-server system. With the first call the dll would invoke the server or the service and pass the call as a request to that server. The server would return the results to the dll and after the call returned to the app, the server/service was still keeping running waiting for more requests from the client dll. The communication between dll and the server/service can be made by any p2p communications, e. g. sockets, pipes, shared memory, ...

Regards, Alex


Ok, here's where I'm at.  I've done this so far to exit the thread.  This keeps the faucet (REQMKTDATA) turned on indefinitely in the background thread called datathread.  If we are disconnected or receive a disconnectflag it terminates gracefully.  Am I correct that simply returning from the thread cleans up the thread BETTER than any endthread, exitthread type of statement??

The problem I now have is that even though the MAIN thread of the program is accessible now for other calls from tradestation (which there are 5-10 more that are based upon the worker background thread) it still LOCKS up tradestation IF that worker thread is not exited gracefully by giving it the disconnectflag = 1.  For example when I program within my tradestation easy language code then 3 minutes after it turns on with md_connect it then turns off with MD_disconnect which only sets the disconnectflag = 1.
if LastBarOnChart and runonce2 = 0 and (connecttime <> 0 and timetominutes(time) >= timetominutes(connecttime) + 3) then begin
      Rtndis= MD_disconnect( IP, inptconnectID, symbolnam, exp, sectype, currency, exchange );
      runonce2 = 1;
Any ideas on how to get around this problem??

Alex, I have to utilize the worker thread in the main thread and have the variables I change in the worker thread (or exe) available always in the main thread of the dll.  For example I need all these functions
int __stdcall MD_nextvalidid ( void  ) {return mdnextValidId;}
double __stdcall MD_open ( void  ) {return mdopen;}
double __stdcall MD_high ( void  ) {return mdhigh;}
double __stdcall MD_low ( void  ) {return mdlow;}
int __stdcall MD_ask ( int indexask  ) {return mdaskvol[indexask];}
int __stdcall MD_bid ( int indexbid  ) {return mdbidvol[indexbid];}

What is the code needed to share between another program (like an exe) and the dll?  I'm going to need a lot more than the suggestion of doing it.  Without concrete examples I will not get it figured out.  For instance despite all the great ideas you all have given here, it was this piece of code I found in the helps that showed me how to create another thread in the first place:

void CheckKey( void *dummy )
    repeat = 0;    /* _endthread implied */


then in the main they had:
 /* Launch CheckKey thread to check for terminating keystroke. */
    _beginthread( CheckKey, 0, NULL );

It was a simple concrete example of how to do it.  
I'm thinking that offloading the reqmktdata to an exe or something AS LONG AS THERE IS NO DIFFERENCE IN WHEN THE VARIABLES CALCULATED IN THE EXE ARE AVAILABLE TO THE DLL, then perhaps that would solve the Tradestation freezing up.  

Thanks again everyone for all their good ideas!!
Just about there.

//flagmdconnect = 1;
		while( EC->IsConnected() && disconnectflag == 0) {
											if (disconnectflag == 1) { 
					delete EC;
			Sleep( 1000 ); 

Open in new window

evilrixSenior Software Engineer (Avast)
>> Am I correct that simply returning from the thread cleans up the thread BETTER than any endthread, exitthread type of statement??
Yes, there is no need to do anything else.


Thanks for all your help, could have used some real world example code in all of this.  Would have liked to implement the dll with exe called but noone helped me with how to share memory space between them
>>>> What is the code needed to share between another program (like an exe) and the dll?  
Sorry, I was busy this week and missed your request.

If you want to go the server way, the dll would

A. start the server process when it was not already running

To check if it was running, the easiest is to check for the existence of a named shared memory which could be used for communication anyhow:
    bool  serverWasRunning = false;
    HANDLE hmap = NULL;
    HANDLE hevent = NULL;
    hMap = CreateFileMapping(                // hMap is the handle returned
               INVALID_HANDLE_VALUE,        // no file but only shared memory
               NULL,                                       // no special access rights
               PAGE_READWRITE,                  // for read and write
               0,                                             // high word of size
               4096,                                       // e. g. 4096 bytes
               "MYSERVER");                          // name of shared memory (use exe name)

    if (hMap != NULL && GetLastError() == ERROR_ALREADY_EXISTS)
          // coming here the server already was started by a previous call
          serverWasRunning = true;
          // we can use the hMap to get a pointer to the memory and
          // put our request to a queue in that memory

    else if (hMap != NULL)
          // the memory  wasn't established yet, so we need to start the server
          STARTUPINFO si = { 0 };
          si.cb = sizeof(si);
          PROCESS_INFORMATION pi = { 0 };

          BOOL ret = CreateProcess(
                                NULL,                                                         // application not used
                                "c:/program files/myserver.exe params",  //  command  
                                NULL, NULL, FALSE,                                   // security .. inheritage
                                DETACHED_PROCESS,                               // no console
                                NULL,                                                         // no special environment
                                "c:/program files",                                     // start directory
                                &si,                                                            // pointer to startup info
                                &pi);                                                          // pointer to process info
          if (!ret)
               return 1;   // create process failed        

          // the server would start and get a handle to the shared memory
          // then, it would wait for an event before starting to process the
          // requests of the input queue established in that memory
          hEvent = CreateEvent(NULL,
                                              FALSE,         // automatic reset
                                              FALSE,         // initial not signaled
                                              "MYSERVER_START");   // unique name
          if (hEvent == NULL)
              return 4;
            // create file mapping failed (rarely)
            return 2;

    // coming here, we have a handle to shared memory and a
    // server waiting for requests
    // get a pointer to shared memory
    void* pMemory = MapViewOfFile(
                                        0, 4096, 4096);

     if (pMemory == NULL)
            return 3;
     // coming here, we can establish the input queue  
     if (!serverWasRunning)
             // we created the shared memory and invoked the server
             // now we have to initialize the memory and let the server run
             memset(pMemory, 0, 4096);
             SetEvent(hEvent);    // sets the signal for the server to go on
             serverWasRunning = true;

     // creates a mutex which initially doesn't claim ownership
     HANDLE hMutex = CreateMutex(NULL, FALSE, "MYCLIENT");
     if (hMutex == NULL)
           return 5;
     // it doesn't matter whether we created the mutex or not we
     // claim exclusive ownership by waiting for it
     // only one client would get ownership at one time
     DWORD dret = WaitForSingleObject(hMutex, 2000);
     if (dret == WAIT_TIMEOUT)
          return 6;    // timeout

     // coming here we safely can write our request to the input queue
     // we then were waiting for the request to be processed by the
     // server

I will add more code if you want to go that way ...

Regards, Alex


Hi Alex,

Well I've been working with this and I see I need to do 2 things.  First I do need to get the exe and dll working together instead of running my connection and marketdata thread in my dll as you suggest.  Before I ask for more of your code, I need to know just HOW I can get my variables in my exe to be read as such in the dll.  For example if I have a variable:
int myvariable = 42; // in the exe
How does the dll know from the shared memory that myvariable = 42??  I guess I need some specific example of sharing variables via this memory space so I can wrap my head around this better and pursue this as it would be much cleaner if I could.
2.  I need to collect more than one data stream (now that I finally got one working I'm thrilled) but each one uses up a clientId and I only have 8 total clientIDs I can use.  Thus, I'm assuming I create some kind of object such that each data stream coming in (identified by TickerID) goes into a different instance of that object.  
So that I have in the final version myobject.myvariable = 42;
Don't know if you are following this or not.
Thanks so much for the help!