<html><body><div style="color:#000; background-color:#fff; font-family:times new roman, new york, times, serif;font-size:12pt"><div>Hi all</div><div><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;">I want to checkpoint an MPI application using MPICH2 and BLCR tool. But while checkpointing i am getting an error.</div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;">I installed BLCR with MPICH2 successfuly. I configured BLCR using command-</div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent;
font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;">./configure --prefix=/home/superusr/manisha/blcr-0.8.3/install<br> make <br>make install</div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;">vim .bash_profile<br><br>export LD_LIBRARY_PATH="/home/superusr/manisha/blcr-0.8.3/install/lib:$LD_LIBRARY_PATH"<br>export PATH="/home/superusr/manisha/blcr-0.8.3/install/bin:$PATH"<br><br>source .bash_profile<br><br>On command prompt Given-<br><br>1) /sbin/insmod /home/superusr/manisha/blcr-0.8.3/install/lib/blcr/2.6.32-279.2.1.el6.x86_64/blcr_imports.ko <br>2)
/sbin/insmod /home/superusr/manisha/blcr-0.8.3/install/lib/blcr/2.6.32-279.2.1.el6.x86_64/blcr.ko<br><br>Checked Whether BLCR is configured or loaded using command -<br><br> /sbin/lsmod | grep blcr<br><br>O/P - <br><br>blcr 114837 0 <br>blcr_imports 9988 1 blcr</div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;">Then i configured the hydra module using command-<br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent;
font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;">tar xzf hydra-1.4.1p1.tar.gz<br><br>cd hydra<br>mkdir hydra-install<br><br>In /home/superusr/manisha/hydra given configure command-<br><br>a. ./configure --prefix=/home/superusr/manisha/hydra/hydra-install --with-hydra-ckpointlib=blcr --with-blcr=/home/superusr/manisha/blcr-0.8.3/install<br>b. make <br>c. make install<br><br>vim .bash_profile<br><br>export LD_LIBRARY_PATH="/home/superusr/manisha/hydra/hydra-install/lib:$LD_LIBRARY_PATH"<br>export PATH="/home/superusr/manisha/hydra/hydra-install/bin:$PATH"<br><br>source .bash_profile</div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new
roman,new york,times,serif; background-color: transparent; font-style: normal;">And then finally i configured MPICH2 using command- <br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;">tar xzf mpich2-1.4.1p1.tar.gz<br><br>cd mpich2-1.4.1p1<br>mkdir mpich2-install<br><br>In /home/superusr/manisha/mpich2-1.4.1p1 given configure command-<br><br>a. ./configure --prefix=/home/superusr/manisha/mpich2-1.4.1p1/mpich2-install --enable-checkpointing --with-blcr=/home/superusr/manisha/blcr-0.8.3/install --with-pm=hydra<br>b. make <br>c. make install<br><br>vim .bash_profile<br><br>export LD_LIBRARY_PATH="/home/superusr/manisha/mpich2-1.4.1p1/mpich2-install/lib:$LD_LIBRARY_PATH"<br>export
PATH="/home/superusr/manisha/mpich2-1.4.1p1/mpich2-install/bin:$PATH"<br><br>source .bash_profile<br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;">And set the hydra checkpoint prefix-<br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;">vim .bash_profile<br><br>export HYDRA_CKPOINT_PREFIX=/home/superusr/Raghu/linpack_10.3.10/benchmarks/mp_linpack/bin_intel/intel64/tmp/app.ckpoint<br><br>source
.bash_profile<br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;">But when i have given the check pointing command as shown below it gives the error given below -</div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;">mpiexec -ckpointlib blcr -ckpoint-prefix /tmp/app.ckpoint -ckpoint-interval 3600 -n 4 ./xhpl_intel64</div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif;
background-color: transparent; font-style: normal;"><br>[cli_2]: Command cmd=put kvsname=kvs_23207_0 key=P0-hostname value=power1.cdacb.in<br> failed, reason='duplicate_keyP0-hostname'<br>[cli_1]: Command cmd=put kvsname=kvs_23207_0 key=P0-hostname value=power1.cdacb.in<br> failed, reason='duplicate_keyP0-hostname'<br>[cli_0]: Command cmd=put kvsname=kvs_23207_0 key=r2h0 value=thnum#1$h0#power1.cdacb.in$r0#0$nbc#1$<br> failed, reason='duplicate_keyr2h0'<br>[cli_3]: Command cmd=put kvsname=kvs_23207_0 key=r2h0 value=thnum#1$h0#power1.cdacb.in$r0#0$nbc#1$<br> failed, reason='duplicate_keyr2h0'<br>[cli_1]: Command cmd=put kvsname=kvs_23207_0 key=P0-businesscard value=fabrics_list#shm$<br> failed, reason='duplicate_keyP0-businesscard'<br>[cli_2]: Command cmd=put kvsname=kvs_23207_0 key=P0-businesscard value=fabrics_list#shm$<br> failed, reason='duplicate_keyP0-businesscard'<br>Fatal error in MPI_Init: Other MPI error, error
stack:<br>MPIR_Init_thread(586)....: Initialization failed<br>MPID_Init(171)...........: channel initialization failed<br>MPIDI_CH3_Init(70).......: <br>MPID_nem_init_ckpt(897)..: <br>MPIDI_PG_SetConnInfo(668): PMI_KVS_Put returned -1<br>Fatal error in MPI_Init: Other MPI error, error stack:<br>MPIR_Init_thread(586)....: Initialization failed<br>MPID_Init(171)...........: channel initialization failed<br>MPIDI_CH3_Init(70).......: <br>MPID_nem_init_ckpt(897)..: <br>MPIDI_PG_SetConnInfo(668): PMI_KVS_Put returned -1</div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;">Please help me resolve this problem.</div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new
york,times,serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;">Thanks & Regards</div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;">Manisha Chauhan<br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0);
font-size: 16px; font-family: times new roman,new york,times,serif; background-color: transparent; font-style: normal;"><br></div></div></body></html>