I want to write a program that will allocate some chunk of memory every second, and then write out how much memory is left (I intentionally do not free any memory as the actual use case for this is simulating out of memory situations). Here is the program (for linux):
#include <stdio.h>
#include <sys/types.h>
#include <unistd.h>
#include <stdlib.h>
int main() {
size_t mbs = 500 * 1000000; // 500mb
while(true) {
sleep(1);
void* tmp = malloc(mbs);
system("free");
}
return 0;
}
The system call to free
lists out the available memory, and it was my expectation that the available memory would decrease every second by the 500mb I am trying to allocate, however as can be seen in first few lines of output below, the available memory does not change in a significant way (out of around 32gb of RAM, the available memory hovers around 25gb). I have also tried lower amounts, 10mb and 100mb but the behavior is the same.
So in this particular output, by the time of the 8th free I would expect 8gbs of ram to be already used:
user@DESKTOP-DAGF175:/Projects/tmp$ g++ main.cpp
user@DESKTOP-DAGF175:/Projects/tmp$ ./a.out
total used free shared buff/cache available
Mem: 33471660 7338384 25903924 17720 229352 25999544
Swap: 8192000 0 8192000
total used free shared buff/cache available
Mem: 33471660 7330684 25911624 17720 229352 26007244
Swap: 8192000 0 8192000
total used free shared buff/cache available
Mem: 33471660 7327256 25915052 17720 229352 26010672
Swap: 8192000 0 8192000
total used free shared buff/cache available
Mem: 33471660 7325428 25916880 17720 229352 26012500
Swap: 8192000 0 8192000
total used free shared buff/cache available
Mem: 33471660 7325208 25917100 17720 229352 26012720
Swap: 8192000 0 8192000
total used free shared buff/cache available
Mem: 33471660 7321232 25921076 17720 229352 26016696
Swap: 8192000 0 8192000
total used free shared buff/cache available
Mem: 33471660 7321360 25920948 17720 229352 26016568
Swap: 8192000 0 8192000
total used free shared buff/cache available
Mem: 33471660 7318624 25923684 17720 229352 26019304
Swap: 8192000 0 8192000
I have also tried a similar thing on windows, but the result is the same. Here is also the windows code:
#include <stdio.h>
#include <stdlib.h>
#include <windows.h>
int main() {
size_t mbs = 500 * 1000000; // 500mb
while(true) {
Sleep(2);
void* tmp = malloc(mbs);
system("systeminfo |find \"Available Physical Memory\"");
}
return 0;
}
Why am I not seeing a decrease in consumed memory?
>Solution :
The behaviour you are observing is due to the way modern operating systems handle memory allocations. When you request memory using malloc()
, the operating system reserves virtual memory addresses but doesn’t immediately allocate physical memory pages.
Only when the program writes to the memory will the operating system map those virtual addresses to physical memory pages.
This behaviour is called lazy allocation or overcommitment. The OS promises the process some memory without really giving it until it is absolutely necessary.
If you want to observe a decrease in the available memory, you need to force the operating system to commit those pages by writing to the allocated memory:
const int PAGE_SIZE = 4096; // commonly used page size
int main() {
size_t mbs = 500 * 1000000; // 500mb
while(true) {
sleep(1);
char* tmp = (char*)malloc(mbs);
if(tmp == NULL) {
printf("Allocation failed\n");
exit(1);
}
// Force the OS to commit the memory by writing to it
for(size_t i = 0; i < mbs; i += PAGE_SIZE) {
tmp[i] = 0;
}
system("free");
}
return 0;
}