[Darshan-users] compiling statically with intel toolchain

[Phil Carns]

Hi Rene,

Does mpiifort support the "-show" option (or something similar) to 
display the underlying link command that it is using?

I can describe the manual steps needed to add Darshan support just to 
see if you can get it working, and then we can maybe circle back to why 
the Darshan-generated script is triggering dynamic linking.  In general, 
there are two sets of extra link flags that Darshan needs to add to the 
link command.  Some should appear before the user-specified libraries in 
the true link command, some should appear after them.  On my test box 
they happen to look like this:

pcarns at carns-x1:~/working/install/bin$ darshan-config --pre-ld-flags
-L/home/pcarns/working/install/lib -ldarshan -lz 
-Wl,@/home/pcarns/working/install/share/ld-opts/darshan-base-ld-opts

pcarns at carns-x1:~/working/install/bin$ darshan-config --post-ld-flags
-L/home/pcarns/working/install/lib -Wl,--start-group -ldarshan 
-ldarshan-stubs -Wl,--end-group -lz -lrt -lpthread

On most MPI implementations there is also one more library that you have 
to add for MPI function wrapping to work.  Darshan-config doesn't add 
that because it is dependent on the MPI implementation. On most versions 
of MPI this is either -lmpifort or -lfmpich.  I'm not sure which one 
Intel MPI uses now.

You might be able to do all of that external to mpiifort, but I'm not 
sure.  If so, then your link command would look something like this 
(possibly swapping out -lmpifort for -lfmpich or omitting it entirely):

mpiifort -DLinux -Dx86_64 -DINTEL -DCompLevel16 -DX86_64 -D__x86_64__ -static-intel -static_mpi -DOpenMP -openmp -DSWAP -O3 -g -o ./ddswriter.revUnversioned -lmpifort `darshan-config --pre-ld-flags` PartitionLayout_mod.o ddswriter.o -L/tstapps/asi/src/dds/lib/Linux/3.0/x86_64/Intel-16.0.1.150MPI5.1.2MKL11.3.1 -ldds_r3 -lgio -lfhost `darshan-config --post-ld-flags`


If that doesn't work then you'll need to find out what the underlying 
link command looks like (via -show or similar) and insert it there.

thanks,
-Phil

On 05/18/2016 05:21 PM, Salmon, Rene wrote:
> Hi,
>
>
>>   I think this means you are using dynamic linking. You could try darshan using the dynamic linking method that uses LD_PRELOAD.
>>
>> http://www.mcs.anl.gov/research/projects/darshan/docs/darshan-runtime.html#_instrumenting_dynamically_linked_fortran_applications
> I tried LD_PRELOAD and that did not do much for me.
>
> We actually want to link in MPI and Darshan statically.  We typically  link MPI statically with intel MPI by just adding the “-static_mpi” option to mpiifort.  Unfortunately the darshan script calls ifort directly and as far as I can tell it is linking in MPI dynamically.  What would I need to add to my link line to get darshan compiled statically?  Here is my link line:
>
> mpiifort -DLinux -Dx86_64 -DINTEL -DCompLevel16 -DX86_64 -D__x86_64__ -static-intel -static_mpi -DOpenMP -openmp -DSWAP -O3 -g -o ./ddswriter.revUnversioned PartitionLayout_mod.o ddswriter.o -L/tstapps/asi/src/dds/lib/Linux/3.0/x86_64/Intel-16.0.1.150MPI5.1.2MKL11.3.1 -ldds_r3 -lgio -lfhost
>
>
>>   Yes, possibly. Intel may have changed their setup but unlikely. I believe the issue here is that that the Intel Fortran MPI functions call the PMPI_* routines directly and thus can not be intercepted unless the specific Fortran interface is intercepted. Do you do I/O from FORTAN directly or are you using an I/O library?
>
> We do all IO using a library written in C. We do not do any IO using fortran.
>
> Thanks for the help
> Rene
>   
>
> _______________________________________________
> Darshan-users mailing list
> Darshan-users at lists.mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/darshan-users