Link to home
Create AccountLog in
Avatar of Peter Chan
Peter ChanFlag for Hong Kong

asked on

Problem with the memory

Hi,
I do get the attached error

using these codes

#include "stdafx.h"
#include <set>
#include <stdio.h>
#include <string.h>
#include <fstream>
#include <string>
#include <ctype.h>
#include <time.h>
#include <process.h>
#include <vector>
#include <iostream>
#include <algorithm>
#include <iomanip>
using namespace std;


struct nameval
{
	char fld_nm[100];
	wchar_t fld_nm_t[100];
     bool operator< (const nameval & a2) const
     {
           if(strcmp(fld_nm, a2.fld_nm) < 0) return true;
           if(strcmp(fld_nm, a2.fld_nm) > 0) return false;
           if (fld_val < a2.fld_val) return true;
           return false;
     }
	int fld_len;
	int fld_val;
};
nameval binrec;

int main()
{
    std::set<nameval> records;
    int cnt;
	for (cnt=0;cnt<8000000;cnt++)
	{
		nameval val={0};
		int j;
		for (j=0;j<20;j++)
		{
			val.fld_nm[j] += (char)(rand () % 58 + 64);
		}
		
		wchar_t wbuf[100]={0};
		mbstowcs(wbuf,val.fld_nm,_countof(val.fld_nm));
		wmemcpy(val.fld_nm_t,wbuf,100);

		val.fld_val=cnt;
		records.insert(val);
	}

	std::ofstream ostrm("c:\\dp4\\flout.bin", std::ios::binary | std::ios::out );
	if (ostrm.is_open())
	{
		for (std::set<nameval>::iterator it = records.begin(); it != records.end(); ++it)
		{
			ostrm.write((char *)&records, sizeof(nameval));
		}
	}

	std::wofstream ostrm2("c:\\dp4\\flout.ord", std::ios::out );
	if (ostrm2.is_open())
	{
		for (std::set<nameval>::iterator it = records.begin(); it != records.end(); ++it)
		{
			ostrm2 << "\"" << it->fld_nm << "\" " << it->fld_len << ' '  << it->fld_nm_t << ' '<< it->fld_val << '\0' << '\n';
		}
	}

 	system("pause>null");

	return 0;
}

Open in new window

t867.png
Avatar of Peter Chan
Peter Chan
Flag of Hong Kong image

ASKER

No, I am to confirm if the way is fine to handle very big vector list. Thanks.
SOLUTION
Avatar of chaau
chaau
Flag of Australia image

Link to home
membership
Create an account to see this answer
Signing up is free. No credit card required.
Create Account
Thanks to your advice to do the correction.
I've corrected it above but I still have the memory problem, after having re-built the project.
It is still the same problem.
SOLUTION
Link to home
membership
Create an account to see this answer
Signing up is free. No credit card required.
Create Account
I have this part now
		for (std::set<nameval>::iterator it = records.begin(); it != records.end(); ++it)
		{
			try
			{
				ostrm.write((char *)&it, sizeof(nameval));
			}
			catch (exception& e)
			{
				cout << e.what() << '\n';
			}
		}
		...

Open in new window

but I still encounter the same when running the re-built project.
The problem is most likely when you assign 80 million records to the set. So, pit the try  catch in the first loop
Hi HuaMinChen,

IMO this can't ever work in a 32-bit build (from the screenshot I assume your app is 32-bit) because if you want allocate 8.000.000 elements of size about 300 byte you'll need around 2,4 GB - this is more than a windows 32-bit app usually can allocate (i.e. take a look at http://stackoverflow.com/questions/639540/how-much-memory-can-a-32-bit-process-access-on-a-64-bit-operating-system about this).

I would suggest to change it to build a 64-bit application to avoid this limit.

Hope that helps,

ZOPPO
Thanks all.
Zoppo,
I am using VS 2010 to the project. Can you advise how to adjust the project to x64?
Chaau,
I do add try...catch to 1st for loop within that, and do get repeating "bad allocation" there. How to resolve it? Thanks.
SOLUTION
Link to home
membership
Create an account to see this answer
Signing up is free. No credit card required.
Create Account
Sorry Zoppo.
I am using VS 2010 express. Can I open Configuration manager from that?
Sorry, I can't say exactly coz I don't use 'Express', but I'm nearly 100% sure it exists in VS 2010 Express too. In VS 2010 there are two possibilities (I know) to open it:

- Menu 'Build' -> 'Configration Manager'
- In properties dialog (i.e. Menu 'Project' -> '<project name> Properties') there's a button 'Configuration Manager' at top-right corner.
Addition: There might be more work needed depending on the history of the project. If it was i.e. created in VS 2010 there shouldn't be to do more, but if it's older it might be there are settings where the same path is set for both b platforms allthough they shouldn't.

If you don't plan to further be able to build as 32 bit you can ignore those settings and I would suggest to remove the Win32 platform from the configuration (even possible with 'Configuration Mananger').

But if you want to keep both platforms you should check if everything is fine.

As a first step you should check if this works:

1. select Win32 platform and do a Rebuild All
2. switch to x64 platform and do a Rebuild All too
3. switch back to Win32 platform and do a normal Build
4. stay in Win32 platform and do a Rebuild All again
5. switch to x64 platform again and do a normal Build

If everything is ok the steps 3 and 5 should finish immediate without recompiling anything.

Otherwise please tell then I can show you which settings probably aren't correct ...

ZOPPO
Many many thanks Zoppo.
I now open the original project (that is created using VS 2010 express), within VS 2013 Pro version. Can you tell me how to open configuration manager within VS 2013 Pro version?
Sorry, please disregard my last question.
Zoppo,
The attached one is the screenshot, within Config manager of VS 2010, can you please advise what to adjust on that? Thanks
t868.png
Please again disregard my current question. Sorry.
ASKER CERTIFIED SOLUTION
Link to home
membership
Create an account to see this answer
Signing up is free. No credit card required.
Create Account