[mpich-discuss] Large memory allocations in MPI applications under Linux

[Sudarshan Raghunathan]

Thank you Jed, I will try your approach and see if it works. I suppose
the simplest solution is to set the rlimit per MPI process assuming an
almost equal distribution of the allocation, but this will not work
when one of the ranks has to allocate a lot more than the others and
the total is still near the available physical memory.

Regards,
Sudarshan

2009/4/15 Jed Brown <jed at 59a2.org>:
> A lightweight solution is to set `ulimit -v' in your shell.
> Alternatively, look at setrlimit (2), RLIMIT_AS.  This limits the total
> amount of virtual memory available to your process.  If you try to
> malloc beyond this limit, it will fail (return NULL).
>
> Most kernels are configured to have no problem wildly over-committing
> memory.  I can malloc 10GB on a machine with 4 GB memory and 4 GB swap.
> Clearly I can't actually touch all of that memory, but malloc doesn't
> mind.  This is a feature and disabling overcommitment may cause problems
> for other programs (depending on what else is running on your machine).
>
> Jed
>