[petsc-users] Memory allocation limit in IBM AIX POE

[Satish Balay]

perhaps not related - but on older IBM machines we used to defualt to
using a compiler option -bmaxdata:0x70000000

Without this - the memory allocatable was much lower than the system
ram. I guess you can give this a try..

Satish

On Mon, 21 Nov 2011, Gong Ding wrote:

> Hi,
> This problem has nothing to do with petsc. 
> However, I hope some one knows POE of AIX in this mailing list.
> 
> We now have a small cluster of PPC6.
> The system is AIX6.1, with xlc and POE.
> I found that when compiled with POE(compiled by mpCC_r, call MPI_Init at the beginning of main function, see below), 
> any memory allocation exceeds about 4M will crash.
> 
> The testing code is really simple:
>  
> #define HAVE_MPI
> 
> #ifdef HAVE_MPI
> #include "mpi.h"
> #endif
> 
> #include <iostream>
> #include <vector>
> 
> struct ST
> {
>   int i;
>   int j;
>   int k; 
>   double v;
> };
> 
> 
> int main(int argc, char **argv)
> {
> #ifdef HAVE_MPI
> 	MPI_Init(&argc, &argv);
> #endif
>   
>   std::vector<ST> array;
>   for(size_t i=0; i<300000; i++)
>   {
>     ST st;
>     array.push_back(st);
>   }
>   
> #ifdef HAVE_MPI
>   MPI_Finalize();
> #endif
>   
>   return 0;
> }
> 
> And the makefile:
> ALL: crash_mpi
> 
> crash_mpi: crash_mpi.o
> 	mpCC_r -q64 -brtl -o crash_mpi crash_mpi.o
> 
> crash_mpi.o : crash_mpi.cc
> 	mpCC_r -q64 -qansialias -qrtti=all -c 	crash_mpi.cc
> 
> The code tries to push 300K structure ST into a vector.
> Total memory request is about 6M.
> This code will always crash.
> However, push 200K ST into vector works well.
> 
> It seems POE has some memory limitation here.
> Does anyone know how to cancle this limitation?
> 
> Thanks
> 
> Gong Ding
> 
>  
> 
>