Solved

Volatile global variable compiler warning

Posted on 2010-08-20
39
1,510 Views
Last Modified: 2012-05-10
I have two threads that share a global array variable.  To be thorough and to not get caught by compiler optimizations, I declared the array as volatile.  This also happens with a single thread.

When I reference the pointer to the array in a function, I get the compiler warning listed below (Visual studio 2005 C compiler)

warning C4090: 'function' : different 'volatile' qualifiers

Why would the fact that's it's a volatile global cause a problem?  What am I doing wrong? ( I looked at the MSDN article but that didn't help)

//global
volatile uint8_t g_my_array[10];

int my_func(void)
{
     memset(g_my_array, 0, sizeof(g_my_array));  //this causes the warning
}

I can cast the first argument as (void*) and get rid of the warning, but that just feels wrong.

What's the right way to do this?

Many thanks.

0
Comment
Question by:JohnSantaFe
  • 15
  • 9
  • 9
  • +2
39 Comments
 
LVL 86

Assisted Solution

by:jkr
jkr earned 100 total points
ID: 33489698
Using your code, all I get is

error C2664: 'memset' : cannot convert parameter 1 from 'volatile uint8_t [10]' to 'void *'        Conversion loses qualifiers

And since 'memset()' expects a 'void*' here, the cast is safe. And since 'volatile' is defined as 'used to declare that an object can be modified in the program by something other than statements', a 'void*' cast will not do any harm.
0
 
LVL 32

Expert Comment

by:phoffric
ID: 33489706
"This warning can be caused when a pointer to a const or volatile item is assigned to a pointer not declared as pointing to const or volatile."
    http://msdn.microsoft.com/en-us/library/k77bkb8d.aspx

The warning message is not consistent across vendors. For example in VS 2008 Express, your code generates a different message - not a warning, but an error:

error C2664: 'memset' : cannot convert parameter 1 from 'volatile uint8_t [10]' to 'void *'

In Cygwin, it generates a warning:
warning: passing argument 1 of `memset' discards qualifiers from pointer target type

>> I can cast the first argument as (void*)
I would cast the first argument as (uint8_t *) just to be explicit as to what kind of argument the first argument really is.

These errors/warnings are probably just a way to remind you that the type qualifiers (const and volatile) cause special behavior of the objects of that type. Maybe the compilers are just trying to make sure that you are aware of the type qualifiers, and you are doing exactly what you intend to do by forcing you to cast away the qualifier.
0
 
LVL 40

Expert Comment

by:evilrix
ID: 33489715
>> I have two threads that share a global array variable.  To be thorough and to not get caught by compiler optimizations, I declared the array as volatile.

Rather than accessing you array as a global in each thread make it an instance of the calling function and pass in a pointer to the array as one of the thread arguments. If you do this you don't really need to make it volatile since the array is accessed indirectly via a pointer the compiler should always read the memory (it can't many any assumptions about it).

>> a 'void*' cast will not do any harm.

It might (but doubtful) if this is being done in a thread.
0
 
LVL 40

Expert Comment

by:evilrix
ID: 33489730
>> Rather than accessing you array as a global in each thread make it an instance of the calling function

BTW: why not just initialise your array thus...

volatile uint8_t g_my_array[10] = {0};

???


// Pseudo(ish) code

void thread_proc(void * arg)

{

  uint8_t * p =  (uint8_t *)arg;

}



void thread_spawner()

{

   uint8_t g_my_array[10] = {0}; == this initialises it too!

   spawn_thread(thread_proc, g_my_array) // the pointer isn't safe from modification but the array it points to is

}

Open in new window

0
 
LVL 32

Expert Comment

by:phoffric
ID: 33489736
>> the pointer isn't safe from modification but the array it points to is
 -- Any good reference links?
0
 
LVL 40

Expert Comment

by:evilrix
ID: 33489741
>> Any good reference links?
To what specifically?
0
 
LVL 86

Expert Comment

by:jkr
ID: 33489743
>>>> a 'void*' cast will not do any harm.
>>It might (but doubtful) if this is being done in a thread.

Um, it does not really matter where you do that cast ;o)

If 'memset()' is called from within a thread, it has to be sync'ed somehow, regardless of the cast. The memory address is the same before and after anyway.
0
 
LVL 32

Expert Comment

by:phoffric
ID: 33489769
The global array, I believe, is a form of communication between the two threads, and presumably that is why the author used volatile (and unstated, possibly a mutex).

Is there a MS Windows reference link that supports the idea that if you have
===
spawn_thread(thread_proc, g_my_array)
      // the pointer isn't safe from modification but the array it points to is
}
===
 then w.r.t. g_my_array, then "the array it points to is safe from modification".
0
 
LVL 40

Expert Comment

by:evilrix
ID: 33489776
>> If 'memset()' is called from within a thread, it has to be sync'ed somehow
It's the array that is volatile... to address a volatile array safely you need a volatile pointer. In essence the result of not doing so is undefined.
0
 
LVL 40

Expert Comment

by:evilrix
ID: 33489791
>> The global array, I believe, is a form of communication between the two threads, and presumably that is why the author used volatile

Indeed - that is stated clearly in the question :)

"I have two threads that share a global array variable.  To be thorough and to not get caught by compiler optimizations"

But the point is you don't need to make the array volatile (or even global), you can just pass the address of the array to each thread and it will quite happily read and write both without any caching issues due to the indirection of access via a pointer (the compiler won't cache the values in the addresses being accessed by the pointer. You could go the extra step and make the pointer volatile but that will only tell the compiler not to cache the address the pointer points to, it has no affect on what the pointer dereferences too.

>> Is there a MS Windows reference link that supports the idea that if you have

No idea, I go by my understanding of how things work according to the C++ Standard - not Microsoft's odd interpretation of it :)
0
 
LVL 32

Expert Comment

by:phoffric
ID: 33489819
>> the compiler won't cache the values in the addresses being accessed by the pointer
     This is the part that I was looking for a supporting link.

Oddly enough, C4090 is a C error; the corresponding C++ error is C2440. (Not sure why there are different error numbers - perhaps some differences between C and C++ in the explanations.)
    C++     http://msdn.microsoft.com/en-us/library/sy5tsf8z.aspx
    C          http://msdn.microsoft.com/en-us/library/k77bkb8d.aspx

So, the author is apparently writing in C, instead of C++ (if one were to believe the links).
0
 
LVL 40

Expert Comment

by:evilrix
ID: 33489834
Slightly off-topic but this article has some interesting views on the use of volatile.
http://software.intel.com/en-us/blogs/2007/11/30/volatile-almost-useless-for-multi-threaded-programming/

The use of intrinsic types and volatile is a nightmare. It was not added to the standard to support MT programming, it was added to support signals. Although the two share similar semantics it can be very easy to over volatile (degrading performance unnecessarily) or under do it and end up with caching issues.

Where possible all thread synchronisation should  be done using OS threading primitives of those primitives that come with your favourite threading library.

Welcome to the jungle :)
0
 
LVL 32

Assisted Solution

by:phoffric
phoffric earned 100 total points
ID: 33489895
Agreed, there are jungle-like aspects to MP, volatile, mutex, signals.
>> The use of ... volatile ... was not added to support MT programming, it was added to support signals.

Actually, a major purpose of volatile was to support memory mapped I/O. Stroustrup even gives an terse statement: "A volatile ... an object may change its value in ways not specified by the language so that aggressive optimizations must be avoided. For exampe, a real time clock might be declared:
    extern const volatile long clock;
Two successive reads of clock might give different results.

Item 2 of your link is on topic; but it appears to be naive and incomplete. However, the reader's comments below help to point out some of the problems. If you want to use a shared memory location, then volatile is necessary. The fact that its slower is just part of a trade-off study to determine the best method of communication. The fact that you may need memory barriers is another "jungle" nuisance, which just shows that synchronization between threads is often not trivial and vendor dependent.

In item 2:
        Message[i/10] = 42;
        Ready = 1;

If Message were not volatile (and if the order of instructions were properly memory barrier'd), then the reader of Message[i/10] might still not get a value of 42. If the reader had read Message[i/10] before, then isn't it possible that its value has been safely cached in a register so that the old stale value will be read again?

So, the part about sending a pointer in spawn_thread as a way to remove the need to make the array volatile still escapes me. But since I have not done Windows threading and also never used spawn_thread(), I thought there may be some MS magic going on. I just wanted to check and get this point clarified with a MS link since the author is using VS 2005 C compiler.
0
 
LVL 40

Expert Comment

by:evilrix
ID: 33489945
>> Stroustrup even gives an terse statement
Don't forget volatile was around way before C++ - it was part of the C90 standard.

>> So, the part about sending a pointer in spawn_thread as a way to remove the need to make the array volatile still escapes me.

The problem is threads aren't part of the standard and neither and compiler optimizations so there is no authoritative answer to this. However, since we are accessing the memory via a lever of indirection the compiler shouldn't be using registers to cache the values in those locations. Generally, compilers only cache intrinsic types. This is certainly my experience with both gcc and Visual Studio - can't say it would apply to every compiler.

The use of volatile is a best endevours solution to a problem that C/C++ alone cannot solve safely. This being the case, as I asset above, you should avoid thread synchronisation unless it is done with OS or thread library specific primitives that are guaranteed to work cross-thread without side effects.

Anyone else notice we seemed to have deviated quite badly from the original question? :)

Oh well, it's all good discussion I guess.

0
 
LVL 32

Expert Comment

by:phoffric
ID: 33489977
>>  it was part of the C90 standard
Yes, K&R said it was new and introduced with ANSI C.

>> the compiler shouldn't be using registers to cache the values in those locations

Ok, this is the part where I was taking issue. Take a simple single threaded process. I would have thought that if you wrote:
void foo( int arr[] ) {
...
  x = arr[4];     // line 1000
...
and
  y = arr[4];    // line 1100
}
then the compiler might put arr[4] into a register in line 1000 so that in line 1100, instead of computing the location of arr[4], the compiler might just use a register value (at least so I thought was the behavior of aa good optimized compiler).

>>  solution to a problem that C/C++ alone cannot solve safely
All agreement here - need thread library specific primitives. But the article seems to purport that volatile can be avoided. And maybe that is true for some platforms; but without documentation to support this, it is a call that I could not safely make; how would I guarantee correct behavior to my team for the entire project life-cycle including maintenance. What if a later compiler release wrote different (but legal) assembly code that thwarted our assumptions.
0
 
LVL 32

Expert Comment

by:phoffric
ID: 33490010
**************************

From all the above discussion, it is evident that there are a number of potential issues in using "volatile".

With all the potential pitfalls associated with "volatile", I now believe that the compilers are giving out errors or warnings to keep you honest; that is, to make sure that you knew that this (global) variable was marked "volatile", and so just didn't do an automatic conversion for you.

**************************
0
 
LVL 32

Expert Comment

by:phoffric
ID: 33490049
Note that in ANSI C and in C++, "volatile" is only a hint to the compiler. A compiler may ignore the "volatile" suggestion and optimize away. If you found this to be the case on your platform, then your use of shared variables to communicate between threads may not work.
0
 

Author Comment

by:JohnSantaFe
ID: 33490266
Hi all,

Thanks for the detailed discussion.  It helps to get a deeper understanding of the topic.  I'll take the time later to read through the posts more thoroughly and the links.

In the past I've used the method of passing the pointer to the thread during thread creation.  That worked ok but I ended up passing the pointer or one of the components of the array as an argument to nearly every function. That's why I decided to try making it a global this time.

The array in my example is actually a structure, I was trying to keep it simple.  It contains a few things that don't really change much but I don't know them until run time e.g. an ethernet socket.

memset() isn't the only place this shows up.  I have a handle to a device that gets used in both threads (mutex used for accessing) and I get the same warning from the driver API.  At least now I understand a little better as to why.

Thanks again.
0
 
LVL 53

Accepted Solution

by:
Infinity08 earned 150 total points
ID: 33490770
I notice that there's a lot of confusion about the use of the volatile keyword, and several diverging ideas about what it's good for.

I'd like to add my vote to not relying on volatile when doing multi-threaded programming. It's not what it's designed for, and in the general case, it's not going to do much good. On the contrary, it'll likely slow down your code, and give you a false sense of security.


>> A compiler may ignore the "volatile" suggestion and optimize away.

That's not true. The standard places pretty strict requirements on the use of 'volatile' :

        [5.1.2.3] "At sequence points, volatile objects are stable in the sense that previous accesses are
        complete and subsequent accesses have not yet occurred."

        [6.7.3] "Furthermore, at every sequence point the value last stored in the
        object shall agree with that prescribed by the abstract machine, except as modified by the
        unknown factors mentioned previously."

But you'll note that none of this really helps with multi-threaded programming. Two threads can still access the same variable at the same time (for reading or writing), and will happily step on each other's feet.

In this specific case, the volatile keyword is used with an array. It's worth to know that it's not the array that is volatile, but the items in the array. In other words, the use of the volatile keyword does not guarantee anything about the array as a whole - just about each item in the array separately.

What you DO need to do when multiple threads have access to the same object, is to protect that access with proper multi-threading primitives, like mutexes, barriers, ...


Finally, with regards to the original question, and the reason why you get that error message. It's because :

        [6.7.3] "If an attempt is
        made to refer to an object defined with a volatile-qualified type through use of an lvalue
        with non-volatile-qualified type, the behavior is undefined."

The compiler is entirely right to give an error, because you're trying to do something that has undefined behavior. And that's not something you want, if you want your code to be useful ;)
0
Free Trending Threat Insights Every Day

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

 
LVL 32

Expert Comment

by:phoffric
ID: 33493555
>> not relying on volatile when doing multi-threaded programming
>> protect that access with proper multi-threading primitives
In this thread, there has only been complete agreements with this. :)

Prior to this thread, I never heard of this total reliance on volatile for multi-threaded synchronization. A surf of the web talks about a common misconception in using volatile solely for multi-threading synchronization, which, from my experience, made no sense. Here is one surf example:
====
"There is a misconception that the volatile qualifier will provide the following desired properties necessary for a multithreaded program:

•Atomicity: indivisible memory operations
•Visibility: the effects of a write action by a thread are visible by other threads
•Ordering: Sequences of memory operations by a thread are guaranteed to be seen in the same order by other threads"
     https://www.securecoding.cert.org/confluence/display/seccode/POS03-C.+Do+not+use+volatile+as+a+synchronization+primitive
====

I found this VS 2005 MSDN article having MS specific extensions which may be the source of this viewpoint:
     http://msdn.microsoft.com/en-us/library/12a04hfd(VS.80).aspx
0
 
LVL 32

Expert Comment

by:phoffric
ID: 33493624
>> "volatile" is only a hint to the compiler.
I got this from a Reference Manual (section A8.2 of K&R ANSI C) which is based on "the draft submitted to ANSI on 31-Oct-1988" - "a compiler may ignore these qualifiers". So, perhaps the draft was amended on this topic on its way to C90.

I also checked The C++ Programming Language Special Edition (Stroustrup, June, 2008), section A.7.1: volatile "is a hint ...". But this is for C++, and this compiler is Visual studio 2005 C compiler. I interpreted "hint" to mean that this qualifier could be ignored by the compiler.
0
 
LVL 53

Expert Comment

by:Infinity08
ID: 33494268
>> volatile "is a hint ...". But this is for C++

Yes, I never understood why they chose to confuse things by using the word "hint". The thing is though, that earlier in the standard, the behavior of a volatile object is well defined, so any conforming implementation still has to make sure not to optimize the code too much. Maybe the word "hint" is referring to the fact that some compilers (or some optimization levels) simply don't use optimizations that would interfere with volatile, and when that's the case, the qualifier volatile is "useless", and can be "ignored".


>> I got this from a Reference Manual (section A8.2 of K&R ANSI C) which is based on "the draft submitted to ANSI on 31-Oct-1988"

I can't talk about that (because I've never read either) - just about what the standard says. And the C standard is pretty clear about it.


>> I never heard of this total reliance on volatile for multi-threaded synchronization.

Just to avoid any confusion : I wasn't just talking about total reliance - I was talking about any reliance (even if it's part of other techniques used to protect access to shared data). I see no use for volatile in multi-threaded programming - only disadvantages.
0
 
LVL 40

Expert Comment

by:evilrix
ID: 33494438
>> I see no use for volatile in multi-threaded programming
http://www.devx.com/tips/Tip/15147
0
 
LVL 53

Expert Comment

by:Infinity08
ID: 33494522
I can't see that article without subscribing to that site. What does it say ?

Maybe I've been a bit too brief, and I should clarify my statement a bit ... "I see no reason for the programmer to use volatile when writing multi-threaded code. It might be used in some of the platform's synchronization primitives, depending on how that helps for a specific compiler/platform, but that should be an implementation detail hidden from the programmer. If you use the proper synchronization primitives, and have a good data sharing strategy, then you don't need volatile, because it will not give you any further guarantees."
0
 
LVL 40

Assisted Solution

by:evilrix
evilrix earned 150 total points
ID: 33494529
>> I can't see that article without subscribing to that site.

The volatile specifier disables various optimizations that a compiler might automatically apply and thus introduce bugs. I'll give two examples:

1) Some hardware interfacing applications periodically read values from a hardware port and copy it to a local variable for further processing. A good example is a timer function that reads the current count from an external clock. If the variable is const, the compiler might skip subsequent reads to make the code more efficient and store the result in a CPU register. To force it to read from the port every time the code accesses it, declare the variable const volatile.

2) In multithreaded applications, thread A might change a shared variable behind thread B's back. Because the compiler doesn¿t see any code that changes the value in thread B, it could assume that B's value has remained unchanged and store it in a CPU register instead of reading it from the main memory. Alas, if thread A changes the variable¿s value between two invocations of thread B, the latter will have have an incorrect value. To force the compiler not to store a variable in a register, declare it volatile.

-- Danny Kalev

>>  If you use the proper synchronization primitives, and have a good data sharing strategy, then you don't need volatile

Agreed.
0
 
LVL 53

Expert Comment

by:Infinity08
ID: 33494565
Thanks for posting what the article says :) The second point sounds like they're trying to do inter-thread communication via a global variable. That's risky, and even using volatile won't guarantee that it'll work in all situations. So, the way I see it, that's a bad example (unless you know exactly how your platform/compiler generates code, and can guarantee that the code will never be run on another platform/compiler).

But, I should have been more verbose when making that statement :)
0
 
LVL 32

Expert Comment

by:phoffric
ID: 33496327
>> That's risky, and even using volatile won't guarantee that it'll work in all situations.
   Could you give examples (or links) about "risky" and the poor warranty.
0
 
LVL 32

Expert Comment

by:phoffric
ID: 33496344
>> I can't see that article without subscribing to that site.
I couldn't either see that article. But for many sites, there is a trick. :)

You google on the keywords (in this case the title "Uses of the ‘volatile’ Keyword"). Even though the URL that appears is identical to the posted link, since it came from Google, the forum wants you to read the contents in the hope that you will perform a query while there. Then you get a list of hits and when you choose one, you will see the sign-in/register message.
0
 
LVL 53

Expert Comment

by:Infinity08
ID: 33496569
>>    Could you give examples (or links) about "risky" and the poor warranty.

I thought that had been made pretty clear already, but sure, why not ?

Risks of sharing global data between threads :

(a) threads trying to update the global data at the same time
(b) threads reading a cached version of the global data, rather than reading the latest version
(c) optimization causing code reordering within threads, which can mess up read/write order for the global data
(d) multi-CPU systems, where different CPU's would access the same global data. Different CPU's could see the read/write operations in a different order.

etc.

The proper way to deal with this, is to use the right mix of mutexes, atomic operations and memory barriers (with the possible exclusion of mutexes - depending on your specific needs).
The only place where the 'volatile' keyword could have a positive effect, is for risk (b), but the other mentioned risks complicate things so much, that you simply need the proper synchronization primitives. And so, the volatile keyword becomes useless (since what it was trying to solve is already being covered by a well-placed memory barrier eg.).

And as an added advantage : not using volatile gives the compiler more chances for optimizing your code, potentially making it faster.
0
 
LVL 32

Expert Comment

by:phoffric
ID: 33496929
>> That's risky, and even using volatile won't guarantee that it'll work in all situations.
>> The only place where the 'volatile' keyword could have a positive effect, is for risk (b)
Ok, misunderstanding. I didn't realize you were talking about "Risks of sharing global data between threads". I thought you were talking about volatile not working all the time or that it should never be used in multi-threaded programs.

(And in some situations, it alone, will not solve the problem. In some platforms having shared memory, it is necessary to make the shared memory attribute volatile in order for the loader to turn off HW caching - but that's more of a multi-processing issue rather than a multi-threading issue since the threads are, by definition, in the same address space.)
0
 
LVL 53

Expert Comment

by:Infinity08
ID: 33496952
>> And in some situations, it alone, will not solve the problem.

I'd say in virtually all situations, rather than just some.

If you want to rely on volatile for any use in multi-threaded programming, you need a very deep understanding of your compiler and your platform, and you need to be very sure that the code will never be run on another compiler/platform. In my experience that is rarely the case, so it's best not to use volatile, and to simply make your life easier, and rely on the proper synchronization primitives.
0
 
LVL 32

Expert Comment

by:phoffric
ID: 33496956
>> so it's best not to use volatile, ... and rely on the proper synchronization primitives.
If one was planning on using shared global variables for messaging, then to avoid using "volatile", what other proposal for thread communication are you recommending?
0
 
LVL 53

Expert Comment

by:Infinity08
ID: 33496983
I've got the impression that we're going quite off topic here. Maybe this should be the subject of a new question ?
0
 
LVL 32

Expert Comment

by:phoffric
ID: 33496995
I just wanted to make sure that I was getting the right message; that we all on the same page.

I think you are saying that we should avoid global variable as a messaging mechanism between threads because if choosing that path, then we are forced to use volatile, which has its own pitfalls - requires a deep understanding of the compiler and platform and is unlikely to be portable.

If, for some reason, the program must use a global variable as a messaging mechanism between threads, then you will have to use volatile, but be advised of various issues.

Does that about sum it up?
0
 
LVL 53

Expert Comment

by:Infinity08
ID: 33497005
No. My point is that you are NOT forced to use volatile, because there are much better alternatives in the form of the proper synchronization primitives.

Unless JohnSantaFe wants us to continue this discussion though, I'll keep it at this, since I feel like we're hijacking his question.
0
 

Author Comment

by:JohnSantaFe
ID: 33507787
Actually I appreciate the in depth treatment of the topic.This question has gotten long so I'd suggest a new one for that reason.

I'll try to summarize what I think I learned as a non-expert.

There is basically no advantage using volatile for protection in a multi-threaded application.  There might be a performance disadvantage since the compiler can't optimize as much.  It's more important to use the thread synchronization primitives.

As for the original question casting the function argument is an acceptable way to get rid of the warning.

Thanks again.


0
 

Author Closing Comment

by:JohnSantaFe
ID: 33507822
This was difficult to score.  Some was off topic but very valuable.  I just tried to distribute to the participants.
0
 
LVL 53

Expert Comment

by:Infinity08
ID: 33508091
Nice summary, JohnSantaFe. And sorry for making it difficult for you :) We can sometimes get carried away with a discussion ...
0

Featured Post

What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

Join & Write a Comment

Suggested Solutions

When writing generic code, using template meta-programming techniques, it is sometimes useful to know if a type is convertible to another type. A good example of when this might be is if you are writing diagnostic instrumentation for code to generat…
Summary: This tutorial covers some basics of pointer, pointer arithmetic and function pointer. What is a pointer: A pointer is a variable which holds an address. This address might be address of another variable/address of devices/address of fu…
Video by: Grant
The goal of this video is to provide viewers with basic examples to understand and use for-loops in the C programming language.
The viewer will learn additional member functions of the vector class. Specifically, the capacity and swap member functions will be introduced.

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

14 Experts available now in Live!

Get 1:1 Help Now