Visual C++ with Boost::thread

Dear experts,

I am writing a program that uses boost::thread. Each thread performs a task that is idenependent from the other threads. This means, each thread calls a member function of an object that is only used by this thread, such that there are no variables used by multiple threads.

Nevertheless, I have the problem that the program works fine as long as only one thread is used. As soon as more than one thread is used the program never ends. When I look into the Windows task manager it still produces CPU-load but memory usage does not grow up.

The relevant code is:

vector<boost::thread *> *workerThreads = new vector<boost::thread *>;
workerThreads->clear( );

for( int i=0; i < numberOfCores; i++ )
{
	workerThread = new boost::thread( &Anova::monteCarlo, anovas->at( i ) );
  	workerThreads->push_back( workerThread );
}

for( int i=0; i < numberOfCores; i++ )
		workerThreads->at( i )->join( );

Open in new window


where anovas is a vector of pointers to objects of type Anova and montoCarlo is a member function of Anova.

Thanks in advance,

Albert
LVL 1
Albert-GeorgAsked:
Who is Participating?
 
phoffricConnect With a Mentor Commented:
Changing modes from Release to Debug (or vice-versa) can affect race conditions as well as introduce memory corruption issues.

Try using one of these Windows tools to find the problem. These windows tools allow a free evaluation period. If one of them helps you, you should purchase it for future work.

IBM Rational Purify

Parasoft® Insure++ Runtime Analysis and Memory Error Detection for C and C++

Intel® Inspector XE 2013

Alternatively, if the rest of your program is portable, then you can run it on a Linux box for testing with the free product:

Helgrind: a thread error detector

which is part of the Valgrind suite of error checking tools.
0
 
evilrixSenior Software Engineer (Avast)Commented:
So, each thread has its own instance of Anova, each being an instance pulled from the vector?

What does Anova::monteCarlo do? It's possible you have a dead-lock in that method but without seeing it it's hard to say.
0
 
Albert-GeorgAuthor Commented:
Yes, each thread has its own instance of Anova.

I have seen now, that it works in debug mode. Maybe, there is a problem with the libraries.
0
Free Tool: Path Explorer

An intuitive utility to help find the CSS path to UI elements on a webpage. These paths are used frequently in a variety of front-end development and QA automation tasks.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

 
evilrixSenior Software Engineer (Avast)Commented:
Again: What does Anova::monteCarlo do? Without seeing this code the best I can do is speculate.

FWIW, there is unlikely to be anything wrong with boost::thread. I use it all the time without such issues.
0
 
Albert-GeorgAuthor Commented:
Anova::monteCarlo does a set of matrix operations. The complete code would be too complicated to post here. It only calls methods and variables of Anova.
0
 
ZoppoCommented:
Hi Albert-Georg,

I would anyhow suggest you post the code of Anova::monteCarlo here, otherwise IMO there's no real way to help you.

As evilrix mentioned it's hard to believe the problem is within boost::thread, so most probably the problem comes from Anova::monteCarlo.

You wrote it works in DEBUG but doesn't work in RELEASE, so a possible reason could be an uninitialized numeric variable - usually uninitialized numeric variables are set to 0 in DEBUG builds but not in RELEASE builds, i.o.w. uninitialized variables can have any arbitrary, unpredictable value in RELEASE builds.

ZOPPO
0
 
evilrixSenior Software Engineer (Avast)Commented:
>> I have seen now, that it works in debug mode.
Are you using a global variable to signal the theads to stop? May be a bool or something like that? If so and if it isn't declared as volatile it is almost certainly optimized out in the threads that are reading it.

http://www.drdobbs.com/cpp/volatile-the-multithreaded-programmers-b/184403766

Better still, you should be using either a boost::condition_variable or a std::atomic (if you are using c++11) variable.

Again, without seeing the code this is just conjecture (but the most likely candidate given the symptoms you've described).
0
 
phoffricCommented:
I have used the Parasoft Insure++ product on legacy code that started to fail after an OS upgrade with great success.
0
 
Albert-GeorgAuthor Commented:
Dear experts,

thank you very much for the valuable tips. I do not use a global variable in order to stop the threads. Anova::monteCarlo contains a for-loop with a fixed number of loop passes.

I will try one of the thread-debugging tools phoffric recommended.

Sincerely,

Albert
0
 
Albert-GeorgAuthor Commented:
I have done further debugging and have found the critical code. Anova::monteCarlo calls a method for computing the pseudoinverse of a matrix. In order to do so a (C)Lapack routine is called. Since Clapack is not C++ I think this could be the problem with multiple threads.

The relevant code is:

void LinAlg::pseudoInverse( double* A, double* Apinv, integer rowsA, integer columnsA )
{
	// A is m * n
	// Apinv is n * m
	integer m = rowsA;
	integer n = columnsA;
	
	// Singular value decomposition of Matrix A
	double *s = new double[min( m, n )];
	double *U = new double[m*m];
	double *Vt = new double[n*n];
	double *Temp = new double[n*m];

	svd( A, s, U, Vt, m, n, true );

	
	// Computing the pseudo-inverse of S (which is a diagonal matrix containing the singular values of s): Spinv (is n x m)
	double *Spinv = new double[n*m];
	int svIndex = 0; // index of vector s

	for( int i=0; i < (n*m); i++ )
	{
		if( i % (n+1) == 0 ) // main diagonal
		{
			if( fabs( s[svIndex] ) > ZERO_TOLERANCE ) // non-zero values of s are inverted
				Spinv[i] = 1.0f / s[svIndex];
			else // zero-values of s
				Spinv[i] = 0.0f;
			svIndex++;
		}
		else // off main diagonal
			Spinv[i] = 0.0f;
	}


	// Compute Apinv = V * Spinv * U'
	multiply( Vt, Spinv, Temp, n, n, n, m, true, false ); // transpose Vt
	multiply( Temp, U, Apinv, n, m, m, m, false, true ); // transpose U; Temp is n x m


	SAFE_DELETE_ARRAY( s );
	SAFE_DELETE_ARRAY( U );
	SAFE_DELETE_ARRAY( Vt );
	SAFE_DELETE_ARRAY( Spinv );
	SAFE_DELETE_ARRAY( Temp );
}

Open in new window


As soon as array s is deleted an error occurs in free.c in the function HeapFree.

So my question is: May classicak C-code (like the one of Clapack) called by multiple threads of Anova::monteCarlo and if not, what would be a thread-safe way to rewrite it?

Thanks,

Albert
0
 
evilrixSenior Software Engineer (Avast)Commented:
>> Since Clapack is not C++ I think this could be the problem with multiple threads.
Unlikely to make any difference since C++ (at least up until the C++11 version) has no specific knowledge about threads. Even if it did (C++11 does), it's still just another thread of execution and providing the code has been written to be thread safe (no race conditions and so on) it should just work.

>> May classicak C-code (like the one of Clapack) called by multiple threads
Yes.

>> As soon as array s is deleted an error occurs in free.c in the function HeapFree.
Are you using a build that is using a C/C++ runtime that is has multi-threaded support? The standard heap allocation and deallocation routines are *NOT* thread-safe (again, because neither C not C++ [before C++11] know anything about threads]. Unless you are using a MT version of the runtime your could have a race condition when calling heap functions.
0
 
Albert-GeorgAuthor Commented:
These are good news. Yes, I use an MT version of the runtime.
0
 
evilrixSenior Software Engineer (Avast)Commented:
Hmmm. Then I'm afraid I'm all out of ideas at this point. I'd be tempted to use the debugger to break at that point in the code and to check the state of the variables. I know this only happens in release but as long as you build the release with symbols you can still debug it (although as the code will be optimized you may find the debugger jumps around a little).

MT issues are very hard to diagnose, especially when you can't see all the code and can't look at things in a debugger. To simplify, I'd just spin up two threads (make sure you still see the problem) and then try and diagnose. This will be significantly easier than trying to debug with many threads.

I also note you are passing pointers (function arguments) around that are non-const. I would make these const to ensure the memory they point to isn't being modified without you realising in multiple threads. This will not be possible if these pointers are const.
0
 
Albert-GeorgAuthor Commented:
Dear experts,

thank you very much again for your help. I tested my code with Intel Inspector and found out that some of the libraries I used were not thread-save. Variables were not initialized and static variables were used. Now it seems to work correctly.

Sincerely,

Albert
0
 
phoffricCommented:
>> Now it seems to work correctly.
      If you have the luxury of extra time, try using another tool to see if you can catch other latent errors. Or, at least, keep running the Intel Inspector on a regular basis, since race conditions that may exist may sporadically cause other errors to crop up. The benefit of these tools is their ability to catch run-time errors. But that means that you have to exercise your programs long enough with a variety of inputs to catch more errors. To catch more unusual errors, you can even do Fuzz Testing with the Inspector watching. You should be able to apply the Inspector to different sections of your program at a time if your program starts to run too slowly.
0
 
Albert-GeorgAuthor Commented:
Thank you for these tips. I bought the inspector in order to be able to do regular testing. It is really a great tool!
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.