[mpich-discuss] Hydra invalid pointer when HYDRA_BOOTSTRAP_EXEC set

[Pavan Balaji]

Seems to work fine for me:

% HYDRA_BOOTSTRAP_EXEC=rsh mpiexec -n 6 ./examples/cpi
Process 5 of 6 is on snowflake
Process 1 of 6 is on snowflake
Process 4 of 6 is on snowflake
Process 2 of 6 is on snowflake
Process 3 of 6 is on snowflake
Process 0 of 6 is on snowflake
pi is approximately 3.1415926544231239, Error is 0.0000000008333307
wall clock time = 0.001801

  -- Pavan

On 01/31/2011 02:11 AM, Steve Sutphen wrote:
> I am trying to get mpich2-1.3.1 working so that I can set system wide
> environment variables that make it easier for the user community to use
> the system.  To that end I would like to set HYDRA_BOOTSTRAP_EXEC=rsh
> (which works on our cluster) but if I set that environment variable
> then mpiexec fails on exit (i.e. it does the computation and
> then catches an invalid pointer being passed to munmap_chunk).
> If I unset the HYDRA_BOOTSTRAP_EXEC variable and add "-bootstrap rsh"
> to the mpiexec parameters it exits clean.
>
> I also downloaded and built mpich2-1.3.2rc1 to see if the problem was
> fixed, but I got the same result.
>
> The system is an older Fedora Core release 5 (Bordeaux), but the
> build did not show any errors so I did not expect that there would be
> a problem.
>
> Thank you,
> 	steve.
>
> Here is the output that I see when it fails:
>
> :mpich2-1.3.1# mpiexec -n 6 ./examples/cpi
> Process 2 of 6 is on eureka002
> Process 4 of 6 is on eureka003
> Process 0 of 6 is on eureka001
> Process 1 of 6 is on eureka001
> Process 5 of 6 is on eureka003
> Process 3 of 6 is on eureka002
> pi is approximately 3.1415926544231243, Error is 0.0000000008333312
> wall clock time = 0.043777
> *** glibc detected *** mpiexec: munmap_chunk(): invalid pointer: 0x00007fffb0b89d3a ***
> ======= Backtrace: =========
> /lib64/libc.so.6(__libc_free+0x17a)[0x3f6ed6d58a]
> mpiexec[0x424abb]
> mpiexec[0x409000]
> mpiexec[0x4058b3]
> /lib64/libc.so.6(__libc_start_main+0xf4)[0x3f6ed1c784]
> mpiexec[0x4042d9]
> ======= Memory map: ========
> 00400000-0044e000 r-xp 00000000 03:03 377538       /opt/Mpich2/bin/mpiexec.hydra
> 0054e000-00550000 rw-p 0004e000 03:03 377538       /opt/Mpich2/bin/mpiexec.hydra
> 00550000-005bb000 rw-p 00550000 00:00 0            [heap]
> 3f6eb00000-3f6eb1a000 r-xp 00000000 03:01 367489   /lib64/ld-2.4.so
> 3f6ec19000-3f6ec1a000 r--p 00019000 03:01 367489   /lib64/ld-2.4.so
> 3f6ec1a000-3f6ec1b000 rw-p 0001a000 03:01 367489   /lib64/ld-2.4.so
> 3f6ed00000-3f6ee3f000 r-xp 00000000 03:01 367490   /lib64/libc-2.4.so
> 3f6ee3f000-3f6ef3e000 ---p 0013f000 03:01 367490   /lib64/libc-2.4.so
> 3f6ef3e000-3f6ef42000 r--p 0013e000 03:01 367490   /lib64/libc-2.4.so
> 3f6ef42000-3f6ef43000 rw-p 00142000 03:01 367490   /lib64/libc-2.4.so
> 3f6ef43000-3f6ef48000 rw-p 3f6ef43000 00:00 0
> 3f6f000000-3f6f080000 r-xp 00000000 03:01 367491   /lib64/libm-2.4.so
> 3f6f080000-3f6f17f000 ---p 00080000 03:01 367491   /lib64/libm-2.4.so
> 3f6f17f000-3f6f180000 r--p 0007f000 03:01 367491   /lib64/libm-2.4.so
> 3f6f180000-3f6f181000 rw-p 00080000 03:01 367491   /lib64/libm-2.4.so
> 3f6f200000-3f6f204000 r-xp 00000000 03:01 99847    /usr/lib64/libnuma.so.1
> 3f6f204000-3f6f304000 ---p 00004000 03:01 99847    /usr/lib64/libnuma.so.1
> 3f6f304000-3f6f305000 rw-p 00004000 03:01 99847    /usr/lib64/libnuma.so.1
> 3f6fc00000-3f6fc14000 r-xp 00000000 03:01 49089    /usr/lib64/libz.so.1.2.3
> 3f6fc14000-3f6fd13000 ---p 00014000 03:01 49089    /usr/lib64/libz.so.1.2.3
> 3f6fd13000-3f6fd14000 rw-p 00013000 03:01 49089    /usr/lib64/libz.so.1.2.3
> 3f6fe00000-3f6fe12000 r-xp 00000000 03:01 368614   /lib64/libpthread-2.4.so
> 3f6fe12000-3f6ff12000 ---p 00012000 03:01 368614   /lib64/libpthread-2.4.so
> 3f6ff12000-3f6ff13000 r--p 00012000 03:01 368614   /lib64/libpthread-2.4.so
> 3f6ff13000-3f6ff14000 rw-p 00013000 03:01 368614   /lib64/libpthread-2.4.so
> 3f6ff14000-3f6ff18000 rw-p 3f6ff14000 00:00 0
> 3f71600000-3f7160d000 r-xp 00000000 03:01 367492   /lib64/libgcc_s-4.1.1-20060525.so.1
> 3f7160d000-3f7170d000 ---p 0000d000 03:01 367492   /lib64/libgcc_s-4.1.1-20060525.so.1
> 3f7170d000-3f7170e000 rw-p 0000d000 03:01 367492   /lib64/libgcc_s-4.1.1-20060525.so.1
> 3f74000000-3f74013000 r-xp 00000000 03:01 368552   /lib64/libnsl-2.4.so
> 3f74013000-3f74113000 ---p 00013000 03:01 368552   /lib64/libnsl-2.4.so
> 3f74113000-3f74114000 r--p 00013000 03:01 368552   /lib64/libnsl-2.4.so
> 3f74114000-3f74115000 rw-p 00014000 03:01 368552   /lib64/libnsl-2.4.so
> 3f74115000-3f74117000 rw-p 3f74115000 00:00 0
> 3f75100000-3f7522b000 r-xp 00000000 03:01 66445    /usr/lib64/libxml2.so.2.6.23
> 3f7522b000-3f7532a000 ---p 0012b000 03:01 66445    /usr/lib64/libxml2.so.2.6.23
> 3f7532a000-3f75337000 rw-p 0012a000 03:01 66445    /usr/lib64/libxml2.so.2.6.23
> 3f75337000-3f75338000 rw-p 3f75337000 00:00 0
> 3f75800000-3f75807000 r-xp 00000000 03:01 368615   /lib64/librt-2.4.so
> 3f75807000-3f75906000 ---p 00007000 03:01 368615   /lib64/librt-2.4.so
> 3f75906000-3f75907000 r--p 00006000 03:01 368615   /lib64/librt-2.4.so
> 3f75907000-3f75908000 rw-p 00007000 03:01 368615   /lib64/librt-2.4.so
> 2aaaaaaab000-2aaaaaaac000 rw-p 2aaaaaaab000 00:00 0
> 2aaaaaacd000-2aaaaaad0000 rw-p 2aaaaaacd000 00:00 0
> 2aaaaaad0000-2aaaaaad2000 r-xp 00000000 03:01 367648 /lib64/libdl-2.4.so
> 2aaaaaad2000-2aaaaabd2000 ---p 00002000 03:01 367648 /lib64/libdl-2.4.so
> 2aaaaabd2000-2aaaaabd3000 r--p 00002000 03:01 367648 /lib64/libdl-2.4.so
> 2aaaaabd3000-2aaaaabd4000 rw-p 00003000 03:01 367648 /lib64/libdl-2.4.so
> 2aaaaabd4000-2aaaaabd6000 rw-p 2aaaaabd4000 00:00 0
> 2aaaaabd6000-2aaaaabe0000 r-xp 00000000 03:01 367886 /lib64/libnss_files-2.4.so
> 2aaaaabe0000-2aaaaacdf000 ---p 0000a000 03:01 367886 /lib64/libnss_files-2.4.so
> 2aaaaacdf000-2aaaaace0000 r--p 00009000 03:01 367886 /lib64/libnss_files-2.4.so
> 2aaaaace0000-2aaaaace1000 rw-p 0000a000 03:01 367886 /lib64/libnss_files-2.4.so
> 7fffb0b75000-7fffb0b8b000 rw-p 7fffb0b75000 00:00 0  [stack]
> ffffffffff600000-ffffffffffe00000 ---p 00000000 00:00 0 [vdso]
> Aborted
>
> _______________________________________________
> mpich-discuss mailing list
> mpich-discuss at mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss

-- 
Pavan Balaji
http://www.mcs.anl.gov/~balaji