STL locale problem in VisualStudio 2015 Release builds.

Hi everybody,

I'm porting some code from VS 2010 to VS 2015 and found a problem with a functionality which uses a STL stringstream to convert a double to a string using system's locale comma-/thousands-separator.

I was able to break it down to this sample program:
#include <iostream>
#include <sstream>

class my_numpunct : public std::numpunct<char>
{
protected:
	char do_decimal_point() const override
	{
		std::cout << "do_decimal_point()" << std::endl;
		return localeconv()->decimal_point[0];
	}

	char do_thousands_sep() const override
	{
		std::cout << "do_thousands_sep()" << std::endl;
		return localeconv()->thousands_sep[0];
	}
};

std::string bar( double v )
{
	std::locale loc( std::locale( "" ), new my_numpunct() );
	std::stringstream ss;
	std::string s;

	ss.imbue( loc );

	ss << v;
	ss >> s;

	std::cout << s << std::endl;

	return s;
}

int main( int argc, char* argv[] )
{
	setlocale( LC_ALL, "" );
	bar( 1234.56789 );
	return 0;
}

Open in new window

When I compile this code as Debug in VS 2015 the output is (as expected):
do_thousands_sep()
do_decimal_point()
1234,57

Open in new window

But if I compile it as Release in VS 2015 the output is just:
1234.56789

Open in new window

In VS 2010 it works correct in both Release and Debug builds.

Does anyone yet encounter similar problems and could probably help me finding a fix or a workaround?

Thanks in advance,

best regards,

ZOPPO


Addition: I found it seems the problem happens at following code in xlocnum around Line 1409:
if (_Poff != _Count)
	_Groupstring[_Poff] = _Punct_fac.decimal_point();

Open in new window

In the debugger I can see the _Punct_fac seems an instance of my class when I look at its __vptr:
*((TestCpp.exe!std::_Facet_base*)(&(*((TestCpp.exe!std::locale::facet*)(&(*((TestCpp.exe!std::numpunct<char>*)(&(*((TestCpp.exe!my_numpunct*)(&(_Punct_fac))))))))))))).__vfptr,8	0x000000013f6a8978 {TestCpp.exe!const my_numpunct::`vftable'} {0x000000013f6a2ee0 {TestCpp.exe!my_numpunct::`scalar deleting destructor'(unsigned int)}, ...}	void *[0x00000008]
 [0x00000000]	0x000000013f6a2ee0 {TestCpp.exe!my_numpunct::`scalar deleting destructor'(unsigned int)}	void *
 [0x00000001]	0x000000013f6a71fa {TestCpp.exe!std::locale::facet::_Incref(void)}	void *
 [0x00000002]	0x000000013f6a71f4 {TestCpp.exe!std::locale::facet::_Decref(void)}	void *
 [0x00000003]	0x000000013f6a4f20 {TestCpp.exe!my_numpunct::do_decimal_point(void)}	void *
 ...

Open in new window


But for any reason my do_decimal_point isn't called ...
LVL 31
ZoppoAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

sarabandeCommented:
in the docs for the locale constructor (https://msdn.microsoft.com/en-us/library/4121792d.aspx) there is a sample code where they replace an existing Facet as well, though not via the constructor but via use_facet.

perhaps you could try whether the issue could be avoided with this kind of coding.

last resort might be proprietary coding like i did recently:

// Use monetary format with fixed 2 for fraction digits and thousand-separators
MyString Amount::ToString() const 
{
    MyString      str; 
    long long amount  = m_amount;  // the amount type contains the price in cent 
    int           decfrac = 2;  // number of decimals after decimal point/comma
    str << amount;   // my string class has "streaming" capabilities
    int off = (amount < 0)? 1 : 0;
    int len = str.GetLength();
    if (len-off <= decfrac) 
    {
        // NUMDECSEP() gets the decimal character as a const char * from current locale
        // Right, Wid, and Fill are manipulators like std::right, std::setw, and std::setfill
        str.Resize(off) << '0' << NUMDECSEP() 
                                << Right() << Wid((BYTE)decfrac) << Fill('0') 
                                << amount;
    }
    else 
    { 
        int decpos = len-decfrac;
        // THSDNUMDIG() returns thousands separator as const char* from current locale
        str.Replace(decpos, 0, NUMDECSEP()); 
        for (int n = decpos-THSDNUMDIG(); n > off; n -= THSDNUMDIG()) 
            str.Replace(n, 0, NUMGRPSEP()); 
    }
    return str;
} 

Open in new window



Sara
ZoppoAuthor Commented:
Hi Sara,

thanks for the reply, but I fear both ways are no really a suitable solution for me because this would mean to find and change each and every place where we use such numpunct-derived classes in a similar way in our code, I guess this would mean some hundred places within more than 2 million LOC - I can only think about such a workaround when I'm 100% sure there's no other way to get it working without need to find and change all those places.

Further (and IMO this is very important) if possible I would like to know why this happens. I now debugged some hours within this functionality and still can't figure out why the virtual functions aren't called so I'm curious if this could probably be a compiler bug, because if so it would be interesting to know if other functionalities could be bogus too.

Best regards,

ZOPPO
sarabandeCommented:
i have noticed when i read the docs that there is also a numpunct Facet for wchar_t type and wonder whether it could be possible that the compiler could not properly detect the template type from your derived Facet class. that would explain why your function wasn't called at all in release mode. perhaps you could try to make your class a template class as well or check whether other facets are working.

so I'm curious if this could probably be a compiler bug
a wrong format of a decimal number is a bug that probably would be reported within days after a new release. even if most projects will not using a new compiler, the big companies would at least do some tests and i can't believe that such an issue could pass unnoticed for a longer period.

at home i have downloaded visual studio 2015 community, though not installed yet. i will try to find time to test your code. i assume it is a win32 console project with multi-byte character set and precompiled header disabled?

Sara
Big Business Goals? Which KPIs Will Help You

The most successful MSPs rely on metrics – known as key performance indicators (KPIs) – for making informed decisions that help their businesses thrive, rather than just survive. This eBook provides an overview of the most important KPIs used by top MSPs.

ZoppoAuthor Commented:
Hm - I can't see how a 'wchar_t' type could be relevant here since all classes I use are 'char'-based, no std::wstring or std::wstringstream is used.

Anyway, if I use 'class my_numpunct : public std::numpunct<wchar_t>' the result is even wrong and the overridden functions aren't called either.

As far as I can see the main problem is there probably situations can exist where a virtual function is not called allthough it should be, it's not 'only' a wrong decimal point.

It is a simple new create Win32 console project, character set is 'Not Set', it uses precompiled headers but in 'stdafx.h' only one line exists:

   #include "targetver.h"

In Release configuration I also disabled all optimization options, beside this I didn't change any settings from the defaults in both Debug and Release.
sarabandeCommented:
I can't see how a 'wchar_t' type could be relevant here
i didn't meant that the compiler used the wrong Facet but perhaps wasn't able to detect the char type and then did nothing with the argument since it doesn't match any of the existing facets.

you could try

template <class chartype>
class my_numpunct : public std::numpunct<chartype>

and then pass my_numpunct<char> to the constructor. by the way, the new object never was deleted, right?

As far as I can see the main problem is there probably situations can exist where a virtual function is not called allthough it should be
you don't mean that actually, do you? a virtual table could go corrupted or get lost if you do a wrong cast, but not calling a virtual function although it was existing, is not a realistic scenario, in my opinion.

Sara
ZoppoAuthor Commented:
I tried that already without any difference, the (with 'new') created my_numpunc-instance is deleted with the stringstream when it's deleted at the end of the function.

I start thinking the problem comes from within runtime library DLLs. When I change the compile settings for 'Runtime Library' from 'Multi-threaded DLL (/MD)' to 'Multi-threaded (/MT)' the problem disappears. Unfortunateley this is even not a good option for me since it would collide with other projects' settings I think.

I'll ask Microsoft Support tomorrow if they know what could be done.

Thanks anyway,

ZOPPO


PS: I leave office now, so I won't respond on further comments before tomorrow morning ...
sarabandeCommented:
i could verify the issue with vs 2015 vcommunity. your code worked in debug mode but not in release mode. different to your description the decimal string was rounded to two fractional digits after the decimal point.

i added code of msdn to main function:

      
time_t rawtime;
	struct tm * timeinfo;
	char buffer[80];

	struct lconv * lc;

	time(&rawtime);
	timeinfo = localtime(&rawtime);

	int twice = 0;

	do {
		printf("Locale is: %s\n", setlocale(LC_ALL, NULL));

		strftime(buffer, 80, "%c", timeinfo);
		printf("Date is: %s\n", buffer);

		lc = localeconv();
		printf("decimalpoint symbol is: %s\n-\n", lc->decimal_point);

		setlocale(LC_ALL, "de-DE");
	} while (!twice++);

Open in new window


and had no problem to retrieve the correct decimal character.

i support your opinion that to change the /MD to /MT is not really an option. if static variables and multi-threading is responsible for the problem, it is a good chance that either the bug would be solved by ms, or that you could solve the issue by moving the i/o generally into one dll.

this would mean to find and change each and every place where we use such numpunct-derived classes in a similar way in our code, I guess this would mean some hundred places

i am currently in a project where we also have redundant coding because programmers have copied existing code rather than to make it reusable. i made the experience that it is always worth to put some efforts into this and make it clean as such code is simply bad and will cause more issues sooner or later.

Sara

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
ZoppoAuthor Commented:
Hi Sara,

thanks for verifying the issue. And well, it's not really a problem about redudancy, we simply have a big amount of classes with functions like 'toString' and 'fromString', most of these functions are implemented in a similar way.

I sent a report about this to Microsoft, let's see what they say.

Have a nice day,

best regards,

ZOPPO
ZoppoAuthor Commented:
JFI, here's the link to the bug report: https://connect.microsoft.com/VisualStudio/feedback/details/2008662/stl-locale-problem-in-visualstudio-2015-release-builds

And, if you're interested, I found another bug where in some situation simple integer substraction can produce sign error, i.e. '300-250 = -50' - I even reported this: https://connect.microsoft.com/VisualStudio/feedback/details/2029793/calculation-error-with-c-compiler-optimizations-in-vs-2015
ZoppoAuthor Commented:
I agree closing the question this way.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
C++

From novice to tech pro — start learning today.