[mpich-discuss] hydra_pmi_proxy: error while loading shared libraries: libimf.so: cannot open shared object file: No such file or directory

Thu Sep 6 13:27:51 CDT 2012

Following your investigation,

#!/bin/sh
echo $LD_LIBRARY_PATH
ldd /path/to/your/application

returned the correct value for the environment variable:

/usr/local/gromos/md++/lib/:/usr/local/gromos/lib/:/usr/local/boost/lib/:/usr/local/openmm/lib/:/opt/intel/composerxe-2011.3.174/compiler/lib/intel64:/opt/intel/composerxe-2011.3.174/ipp/../compiler/lib/intel64:/opt/intel/composerxe-2011.3.174/ipp/lib/intel64:/opt/intel/composerxe-2011.3.174/compiler/lib/intel64:/opt/intel/composerxe-2011.3.174/mkl/lib/intel64:/opt/intel/composerxe-2011.3.174/tbb/lib/intel64//cc4.1.0_libc2.4_kernel2.6.16.21:/opt/intel/composerxe-2011.3.174/mpirt/lib/intel64:/usr/local/mpich2.shared.exec/lib/

and ldd returned:

        linux-vdso.so.1 =>  (0x00007fff0e2fd000)
        libboost_program_options.so.1.42.0 =>
/usr/local/boost/lib/libboost_program_options.so.1.42.0 (0x00007f6c29f42000)
        libboost_filesystem.so.1.42.0 =>
/usr/local/boost/lib/libboost_filesystem.so.1.42.0 (0x00007f6c29d2c000)
        libboost_thread.so.1.42.0 =>
/usr/local/boost/lib/libboost_thread.so.1.42.0 (0x00007f6c29b17000)
        libmpichcxx.so.3 =>
/usr/local/mpich2.shared.exec/lib/libmpichcxx.so.3 (0x00007f6c298f5000)
        libpmpich.so.3 => /usr/local/mpich2.shared.exec/lib/libpmpich.so.3
(0x00007f6c294b8000)
        libopa.so.1 => /usr/local/mpich2.shared.exec/lib/libopa.so.1
(0x00007f6c292b7000)
        libmpl.so.1 => /usr/local/mpich2.shared.exec/lib/libmpl.so.1
(0x00007f6c290b2000)
        librt.so.1 => /lib/librt.so.1 (0x00007f6c28e8f000)
        libpthread.so.0 => /lib/libpthread.so.0 (0x00007f6c28c72000)
        libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f6c2895e000)
        libm.so.6 => /lib/libm.so.6 (0x00007f6c286da000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00007f6c284c3000)
        libc.so.6 => /lib/libc.so.6 (0x00007f6c28140000)
        libboost_system.so.1.42.0 =>
/usr/local/boost/lib/libboost_system.so.1.42.0 (0x00007f6c27f3b000)
        libimf.so =>
/opt/intel/composerxe-2011.3.174/compiler/lib/intel64/libimf.so
(0x00007f6c27b56000)
        libsvml.so =>
/opt/intel/composerxe-2011.3.174/compiler/lib/intel64/libsvml.so
(0x00007f6c274a2000)
        libintlc.so.5 =>
/opt/intel/composerxe-2011.3.174/compiler/lib/intel64/libintlc.so.5
(0x00007f6c27352000)
        libdl.so.2 => /lib/libdl.so.2 (0x00007f6c2714e000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f6c2a19e000)

which seems correct to me.

As for the

   mpiexec echo \$LD_LIBRARY_PATH

it didn't work here. I only got

  $LD_LIBRARY_PATH

I tested

  mpiexec "echo \$LD_LIBRARY_PATH"  >> $log_file 2>&1
,
  mpiexec echo "\$LD_LIBRARY_PATH"  >> $log_file 2>&1

and

  mpiexec echo \$LD_LIBRARY_PATH  >> $log_file 2>&1

but got the same result every time.
I am not sure how the substitution of environment variables works when
mpiexec is called but it looks like there is no substitution at all, since
I didn't get a blank line, as if the variable was empty or not set, I got
its name.

------------------------------

>
> Message: 4
> Date: Tue, 4 Sep 2012 22:28:56 +0200
> From: Reuti <reuti at staff.uni-marburg.de <javascript:;>>
> To: mpich-discuss at mcs.anl.gov <javascript:;>
> Subject: Re: [mpich-discuss] hydra_pmi_proxy: error while loading
>         shared  libraries: libimf.so: cannot open shared object file: No
> such
>         file or directory (Reuti)
> Message-ID:
>         <1935634D-C475-4A7C-8110-200976C5DDB9 at staff.uni-marburg.de<javascript:;>
> >
> Content-Type: text/plain; charset=us-ascii
>
> Am 04.09.2012 um 22:09 schrieb Marcelo C. R. Melo:
>
> > Yes Reuti, the LD_LIBRARY_PATH is configured correctly.
> > I can use the software normally when logged into any node.
>
> The environment will be different when started by a queuing system, at
> least for SGE I know this for sure and I think also for Torque/PBS. To
> investigate, can you please submit a job with:
>
> #!/bin/sh
> echo $LD_LIBRARY_PATH
> ldd /path/to/your/application
>
> If we know this, we have to check the processes on the slave nodes of the
> parallel job. Usually they are not started by an `ssh`, but some builtin
> mechanism inside the queuingsystem. So, submitting a job with this should
> show it:
>
> #!/bin/sh
> mpiexec echo \$LD_LIBRARY_PATH
>
> It's necessary to escape the $, as we want to get the result from the
> slave, not where the jobscript is executed.
>
> -- Reuti
>
>
> > The problem is that when I Spawn another executable from my initial
> launcher program, the spawned executable does not see the environment that
> the launcher received.
> >
> > Is there any portable way to pass the environment variables to Spawned
> processes?
> >
> > I posted this question on stack overflow a few days ago:
> http://stackoverflow.com/questions/12252490/why-cant-and-environment-variable-be-seen-by-an-executable-if-it-is-run-on-two/12252584#comment16448588_12252584
> >
> > Best regards,
> > Marcelo
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/mpich-discuss/attachments/20120906/1db6b332/attachment.html>