Visual Stdio 2008 最大内存分配块大小问题: 使用new 分配连续723M内存 出错 std::bad_alloc at memory location 0x0013e0b8

 

I am using visual studio 2008 for developing. My program needs to deal with a huge amount of memory. The error happens when my program try to allocate a 512M float array. Code is the following:

int size =512*512*512;
float*buffer =newfloat[size];

Before this allocation, the program already consumed around 554M memory. My desktop has 4G main memory and I am using windows xp 32bits.

How can I avoid the allocation error? Thanks very much for your input!

c++ visual-studio-2008 memory-allocation

	Not a solution, but why aren't you using std::vector? – GMan Sep 13 '10 at 16:55
	How is that 512M? Your variable initialization should be int size = 512*1024*1024;. Also, I'd change the data type of size to size_t. – Prætorian Sep 13 '10 at 17:24
	Is there any reason you need to have a 512M array (128M * sizeof(float))? Can you break that up into smaller chunks? Can you move to a 64-bit system with more memory? – David Thornley Sep 13 '10 at 17:41
	$1.00 says you don't really need a 512 MB float array. – John Dibling Sep 13 '10 at 19:45
	The 512m array is used as a temporary buffer for vector quantization of a 512*512*512 dataset. I can certainly break it into small chunks, just need some extra works. I am targeting a deadline, so the less work the better. :) Thanks for the input. – Aaron Sep 13 '10 at 20:52

feedback

Answer

 

up vote 4 down vote accepted

Your array requires too much contiguous memory. Your program has a bit less of 2 gigabytes of virtual memory available but that address space is broken up by chunks of code, data and various heaps. Memory is allocated from the free space between those chunks. On a 32-bit operating system you can get ~650 MB when you allocate immediately. That goes South when your program starts using memory. The sum of all memory allocations is still ~2GB.

Use a 64-bit operating system or partition your data structures. SysInternals' VMMap utility can give you insight in the virtual memory mapping of your program.

	Using the /LARGEADDRESSAWARE linker switch can get you past the 2GB limit. I think you're fine up to around 3GB. Of course, your point about contiguous memory still holds. – Nathan Ernst Sep 13 '10 at 17:24
	Meh, getting the boot option to work is the rub. – Hans Passant Sep 13 '10 at 17:28
	the software you recommend is pretty good. Thanks. – Aaron Sep 13 '10 at 20:53