[Darshan-users] compiling statically with intel toolchain

Mon May 23 17:21:10 CDT 2016

Rene,

  Your code is making the mpi_init and mpi_finalize call from Fortran correct? You are still hitting the same issue as you noted before which is that the Darshan doesn’t intercept those calls with the Intel MPI implementation. You could specifically add the darshan calls to your Fortran code manually. It isn’t pretty but it should work. This will initialize the library and then it should capture all the calls from your I/O code since that is all in C.

  call mpi_init()
  call darshan_core_initialize(%VAL(0), %VAL(0));
  …
  call darshan_core_shutdown();
  call mpi_finalize()

  depending on the function names, you may need to add an underscore to the end of the names.

kevin

>Hi Phil,
>
>Thanks for the info that was helpful.  I did manage to link in darshan and MPI statically I had to put in the “lib.a” files by hand as the “darshan-config –pre/post” scripts where still trying to link in darshan dynamically. 
>Here is the link line I used that seem to generate a binary.
>
>
>mpiifort  -DLinux -Dx86_64 -DINTEL -DCompLevel16 -DX86_64 -D__x86_64__ -static-intel -static_mpi -DOpenMP -openmp -DSWAP -O3 -g -o /hpcdata/salmr0/mspain/usp2hdf5/dds/bin/Linux/3.0/x86_64/ddswriter.revUnversioned /hpc/tstapps/intel/x86_64/ics-2016.update.1/compilers_and_libraries_2016.1.150/linux/mpi/intel64/lib/libmpifort.a -L/hpc/tstapps/SLES/3.0/x86_64/darshan/3.0.0/lib /hpc/tstapps/SLES/3.0/x86_64/darshan/3.0.0/lib/libdarshan.a -lz -Wl,@/hpc/tstapps/SLES/3.0/x86_64/darshan/3.0.0/share/ld-opts/darshan-base-ld-opts svn_version.o PartitionLayout_mod.o ddswriter.o -L/tstapps/asi/src/dds/lib/Linux/3.0/x86_64/Intel-16.0.1.150MPI5.1.2MKL11.3.1 -ldds_r3 -lgio -lfhost -L/hpc/tstapps/SLES/3.0/x86_64/darshan/3.0.0/lib -Wl,--start-group /hpc/tstapps/SLES/3.0/x86_64/darshan/3.0.0/lib/libdarshan.a /hpc/tstapps/SLES/3.0/x86_64/darshan/3.0.0/lib/libdarshan-stubs.a -Wl,--end-group -lz
>
>
>
>Here is what the binary look like:
>
>hpcbuild02(salmr0)1181:ldd bin/Linux/3.0/x86_64/ddswriter
>	linux-vdso.so.1 =>  (0x00007ffe791a7000)
>	libz.so.1 => /lib64/libz.so.1 (0x00002aafaab7c000)
>	libdl.so.2 => /lib64/libdl.so.2 (0x00002aafaad92000)
>	librt.so.1 => /lib64/librt.so.1 (0x00002aafaaf96000)
>	libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aafab1a0000)
>	libm.so.6 => /lib64/libm.so.6 (0x00002aafab3bd000)
>	libc.so.6 => /lib64/libc.so.6 (0x00002aafab636000)
>	/lib64/ld-linux-x86-64.so.2 (0x00002aafaa95a000)
>	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002aafab9b3000)
>hpcbuild02(salmr0)1182:nm bin/Linux/3.0/x86_64/ddswriter|grep darshan
>0000000000482e90 T darshan_clean_file_path
>0000000000483210 t darshan_common_val_compare
>00000000004830c0 T darshan_common_val_counter
>00000000004831a0 t darshan_common_val_walker
>000000000101e860 b darshan_core
>0000000000463f50 t darshan_core_cleanup
>00000000004655e0 T darshan_core_initialize
>0000000000fcfa80 d darshan_core_mutex
>0000000000464720 T darshan_core_register_module
>0000000000464b80 T darshan_core_register_record
>0000000000461c10 T darshan_core_shutdown
>0000000000464840 T darshan_core_unregister_module
>0000000000465310 T darshan_core_unregister_record
>00000000004655b0 T darshan_core_wtime
>0000000000463e40 t darshan_deflate_buffer..0
>00000000004642b0 t darshan_get_shared_records
>00000000004806a0 T darshan_hash
>0000000000fcfab0 d darshan_mem_alignment
>0000000000fcfaec d darshan_mem_alignment
>0000000000b64848 r darshan_module_names
>0000000000b6487c r darshan_module_versions
>0000000000479ae0 T darshan_mpiio_shutdown_bench_setup
>0000000000fcfa20 D darshan_path_exclusions
>0000000000470250 T darshan_posix_shutdown_bench_setup
>0000000000461750 T darshan_shutdown_bench
>0000000000483230 T darshan_variance_reduce
>0000000000483170 T darshan_walk_common_vals
>
>
>
>The problem I am having now is that I don’t get a darshan log file when I run my executable. I configured darshan with this configure flag “--with-log-path-by-env=DARSHANLOGPATH”.  I set that environment variable in my shell to some directory and then run my executable.  The executable runs fine but no darshan log file gets generated. 
>
>Any ideas?   Thanks in advanced. 
>
>Rene
>
>
>
>
>
>
>On 5/20/16, 7:33 AM, "darshan-users-bounces at lists.mcs.anl.gov on behalf of Phil Carns" <darshan-users-bounces at lists.mcs.anl.gov on behalf of carns at mcs.anl.gov> wrote:
>
>>Hi Rene,
>>
>>Does mpiifort support the "-show" option (or something similar) to 
>>display the underlying link command that it is using?
>>
>>I can describe the manual steps needed to add Darshan support just to 
>>see if you can get it working, and then we can maybe circle back to why 
>>the Darshan-generated script is triggering dynamic linking.  In general, 
>>there are two sets of extra link flags that Darshan needs to add to the 
>>link command.  Some should appear before the user-specified libraries in 
>>the true link command, some should appear after them.  On my test box 
>>they happen to look like this:
>>
>>pcarns at carns-x1:~/working/install/bin$ darshan-config --pre-ld-flags
>>-L/home/pcarns/working/install/lib -ldarshan -lz 
>>-Wl,@/home/pcarns/working/install/share/ld-opts/darshan-base-ld-opts
>>
>>pcarns at carns-x1:~/working/install/bin$ darshan-config --post-ld-flags
>>-L/home/pcarns/working/install/lib -Wl,--start-group -ldarshan 
>>-ldarshan-stubs -Wl,--end-group -lz -lrt -lpthread
>>
>>On most MPI implementations there is also one more library that you have 
>>to add for MPI function wrapping to work.  Darshan-config doesn't add 
>>that because it is dependent on the MPI implementation. On most versions 
>>of MPI this is either -lmpifort or -lfmpich.  I'm not sure which one 
>>Intel MPI uses now.
>>
>>You might be able to do all of that external to mpiifort, but I'm not 
>>sure.  If so, then your link command would look something like this 
>>(possibly swapping out -lmpifort for -lfmpich or omitting it entirely):
>>
>>mpiifort -DLinux -Dx86_64 -DINTEL -DCompLevel16 -DX86_64 -D__x86_64__ -static-intel -static_mpi -DOpenMP -openmp -DSWAP -O3 -g -o ./ddswriter.revUnversioned -lmpifort `darshan-config --pre-ld-flags` PartitionLayout_mod.o ddswriter.o -L/tstapps/asi/src/dds/lib/Linux/3.0/x86_64/Intel-16.0.1.150MPI5.1.2MKL11.3.1 -ldds_r3 -lgio -lfhost `darshan-config --post-ld-flags`
>>
>>
>>If that doesn't work then you'll need to find out what the underlying 
>>link command looks like (via -show or similar) and insert it there.
>>
>>thanks,
>>-Phil
>>
>
>
>_______________________________________________
>Darshan-users mailing list
>Darshan-users at lists.mcs.anl.gov
>https://lists.mcs.anl.gov/mailman/listinfo/darshan-users
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4090 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/darshan-users/attachments/20160523/6c15c0f6/attachment-0001.bin>