Avatar of Peter Chan
Peter Chan
Flag for Hong Kong asked on

Problem with the memory

Hi,
I do get the attached error

using these codes

#include "stdafx.h"
#include <set>
#include <stdio.h>
#include <string.h>
#include <fstream>
#include <string>
#include <ctype.h>
#include <time.h>
#include <process.h>
#include <vector>
#include <iostream>
#include <algorithm>
#include <iomanip>
using namespace std;


struct nameval
{
	char fld_nm[100];
	wchar_t fld_nm_t[100];
     bool operator< (const nameval & a2) const
     {
           if(strcmp(fld_nm, a2.fld_nm) < 0) return true;
           if(strcmp(fld_nm, a2.fld_nm) > 0) return false;
           if (fld_val < a2.fld_val) return true;
           return false;
     }
	int fld_len;
	int fld_val;
};
nameval binrec;

int main()
{
    std::set<nameval> records;
    int cnt;
	for (cnt=0;cnt<8000000;cnt++)
	{
		nameval val={0};
		int j;
		for (j=0;j<20;j++)
		{
			val.fld_nm[j] += (char)(rand () % 58 + 64);
		}
		
		wchar_t wbuf[100]={0};
		mbstowcs(wbuf,val.fld_nm,_countof(val.fld_nm));
		wmemcpy(val.fld_nm_t,wbuf,100);

		val.fld_val=cnt;
		records.insert(val);
	}

	std::ofstream ostrm("c:\\dp4\\flout.bin", std::ios::binary | std::ios::out );
	if (ostrm.is_open())
	{
		for (std::set<nameval>::iterator it = records.begin(); it != records.end(); ++it)
		{
			ostrm.write((char *)&records, sizeof(nameval));
		}
	}

	std::wofstream ostrm2("c:\\dp4\\flout.ord", std::ios::out );
	if (ostrm2.is_open())
	{
		for (std::set<nameval>::iterator it = records.begin(); it != records.end(); ++it)
		{
			ostrm2 << "\"" << it->fld_nm << "\" " << it->fld_len << ' '  << it->fld_nm_t << ' '<< it->fld_val << '\0' << '\n';
		}
	}

 	system("pause>null");

	return 0;
}

Open in new window

t867.png
C++COperating Systems

Avatar of undefined
Last Comment
Zoppo

8/22/2022 - Mon
Peter Chan

ASKER
No, I am to confirm if the way is fine to handle very big vector list. Thanks.
SOLUTION
chaau

Log in or sign up to see answer
Become an EE member today7-DAY FREE TRIAL
Members can start a 7-Day Free trial then enjoy unlimited access to the platform
Sign up - Free for 7 days
or
Learn why we charge membership fees
We get it - no one likes a content blocker. Take one extra minute and find out why we block content.
Not exactly the question you had in mind?
Sign up for an EE membership and get your own personalized solution. With an EE membership, you can ask unlimited troubleshooting, research, or opinion questions.
ask a question
Peter Chan

ASKER
Thanks to your advice to do the correction.
I've corrected it above but I still have the memory problem, after having re-built the project.
Peter Chan

ASKER
It is still the same problem.
Your help has saved me hundreds of hours of internet surfing.
fblack61
SOLUTION
Log in to continue reading
Log In
Sign up - Free for 7 days
Get an unlimited membership to EE for less than $4 a week.
Unlimited question asking, solutions, articles and more.
Peter Chan

ASKER
I have this part now
		for (std::set<nameval>::iterator it = records.begin(); it != records.end(); ++it)
		{
			try
			{
				ostrm.write((char *)&it, sizeof(nameval));
			}
			catch (exception& e)
			{
				cout << e.what() << '\n';
			}
		}
		...

Open in new window

but I still encounter the same when running the re-built project.
chaau

The problem is most likely when you assign 80 million records to the set. So, pit the try  catch in the first loop
Zoppo

Hi HuaMinChen,

IMO this can't ever work in a 32-bit build (from the screenshot I assume your app is 32-bit) because if you want allocate 8.000.000 elements of size about 300 byte you'll need around 2,4 GB - this is more than a windows 32-bit app usually can allocate (i.e. take a look at http://stackoverflow.com/questions/639540/how-much-memory-can-a-32-bit-process-access-on-a-64-bit-operating-system about this).

I would suggest to change it to build a 64-bit application to avoid this limit.

Hope that helps,

ZOPPO
Get an unlimited membership to EE for less than $4 a week.
Unlimited question asking, solutions, articles and more.
Peter Chan

ASKER
Thanks all.
Zoppo,
I am using VS 2010 to the project. Can you advise how to adjust the project to x64?
Peter Chan

ASKER
Chaau,
I do add try...catch to 1st for loop within that, and do get repeating "bad allocation" there. How to resolve it? Thanks.
SOLUTION
Log in to continue reading
Log In
Sign up - Free for 7 days
Get an unlimited membership to EE for less than $4 a week.
Unlimited question asking, solutions, articles and more.
Peter Chan

ASKER
Sorry Zoppo.
I am using VS 2010 express. Can I open Configuration manager from that?
Experts Exchange is like having an extremely knowledgeable team sitting and waiting for your call. Couldn't do my job half as well as I do without it!
James Murphy
Zoppo

Sorry, I can't say exactly coz I don't use 'Express', but I'm nearly 100% sure it exists in VS 2010 Express too. In VS 2010 there are two possibilities (I know) to open it:

- Menu 'Build' -> 'Configration Manager'
- In properties dialog (i.e. Menu 'Project' -> '<project name> Properties') there's a button 'Configuration Manager' at top-right corner.
Zoppo

Addition: There might be more work needed depending on the history of the project. If it was i.e. created in VS 2010 there shouldn't be to do more, but if it's older it might be there are settings where the same path is set for both b platforms allthough they shouldn't.

If you don't plan to further be able to build as 32 bit you can ignore those settings and I would suggest to remove the Win32 platform from the configuration (even possible with 'Configuration Mananger').

But if you want to keep both platforms you should check if everything is fine.

As a first step you should check if this works:

1. select Win32 platform and do a Rebuild All
2. switch to x64 platform and do a Rebuild All too
3. switch back to Win32 platform and do a normal Build
4. stay in Win32 platform and do a Rebuild All again
5. switch to x64 platform again and do a normal Build

If everything is ok the steps 3 and 5 should finish immediate without recompiling anything.

Otherwise please tell then I can show you which settings probably aren't correct ...

ZOPPO
Peter Chan

ASKER
Many many thanks Zoppo.
I now open the original project (that is created using VS 2010 express), within VS 2013 Pro version. Can you tell me how to open configuration manager within VS 2013 Pro version?
Get an unlimited membership to EE for less than $4 a week.
Unlimited question asking, solutions, articles and more.
Peter Chan

ASKER
Sorry, please disregard my last question.
Peter Chan

ASKER
Zoppo,
The attached one is the screenshot, within Config manager of VS 2010, can you please advise what to adjust on that? Thanks
t868.png
Peter Chan

ASKER
Please again disregard my current question. Sorry.
All of life is about relationships, and EE has made a viirtual community a real community. It lifts everyone's boat
William Peck
ASKER CERTIFIED SOLUTION
Log in to continue reading
Log In
Sign up - Free for 7 days
Get an unlimited membership to EE for less than $4 a week.
Unlimited question asking, solutions, articles and more.