[Darshan-users] compiling statically with intel toolchain
Salmon, Rene
salmr0 at bp.com
Tue May 24 09:21:12 CDT 2016
All,
Thank you very much for all the help and info. I now have a working binary that does generate a darshan log file. Your hints where very helpful. What I ended up doing was writing an mpi_init and mpi_finalize wrapper in C that my fortran code calls and that seems to work and now I can generate a darshan log file.
I basically just call my_finalize() from my fortran code and that is just this:
#include "mpi.h"
int my_finalize_(void)
{
int ier;
ier = MPI_Finalize();
return(ier);
}
Now I just want to analyze the results so I run the darshan-job-summary.pl but I am getting these errors. Any ideas what they mean?
hpcbuild02(salmr0)1054:darshan-job-summary.pl ./salmr0_ddswriter_id40315_5-24-32872-5756027388706526063_1.darshan
Slowest unique file time: 0.004726
Slowest shared file time: 0.011293
Total bytes read and written by app (may be incorrect): 51934
Total absolute I/O time: 0.016019
**NOTE: above shared and unique file times calculated using MPI-IO timers if MPI-IO interface used on a given file, POSIX timers otherwise.
Cannot open load file '-e'
line 0: util.c: No such file or directory
EPSTOPDF 2.9.5gw, 2006/01/29 - Copyright 1998-2006 by Sebastian Rahtz et al.
!!! Error: 'posix-access-hist.eps' does not exist!
Cannot open load file '-e'
line 0: util.c: No such file or directory
EPSTOPDF 2.9.5gw, 2006/01/29 - Copyright 1998-2006 by Sebastian Rahtz et al.
!!! Error: 'file-access-read.eps' does not exist!
EPSTOPDF 2.9.5gw, 2006/01/29 - Copyright 1998-2006 by Sebastian Rahtz et al.
!!! Error: 'file-access-write.eps' does not exist!
EPSTOPDF 2.9.5gw, 2006/01/29 - Copyright 1998-2006 by Sebastian Rahtz et al.
!!! Error: 'file-access-shared.eps' does not exist!
LaTeX generation (phase1) failed [17920], aborting summary creation.
error log:
Overfull \hbox (18.76118pt too wide) in paragraph at lines 52--52
[][]
<time-summary.pdf, id=1, 289.08pt x 252.945pt> <use time-summary.pdf>
<op-counts.pdf, id=2, 289.08pt x 252.945pt> <use op-counts.pdf>
LaTeX Warning: File `posix-access-hist.pdf' not found on input line 68.
!pdfTeX error: pdflatex (file posix-access-hist.pdf): cannot find image file
==> Fatal error occurred, no output PDF file produced!
Thanks
Rene
On 5/23/16, 5:21 PM, "Harms, Kevin" <harms at alcf.anl.gov> wrote:
>Rene,
>
> Your code is making the mpi_init and mpi_finalize call from Fortran correct? You are still hitting the same issue as you noted before which is that the Darshan doesn’t intercept those calls with the Intel MPI implementation. You could specifically add the darshan calls to your Fortran code manually. It isn’t pretty but it should work. This will initialize the library and then it should capture all the calls from your I/O code since that is all in C.
>
> call mpi_init()
> call darshan_core_initialize(%VAL(0), %VAL(0));
> …
> call darshan_core_shutdown();
> call mpi_finalize()
>
> depending on the function names, you may need to add an underscore to the end of the names.
>
>kevin
>
>>Hi Phil,
>>
>>Thanks for the info that was helpful. I did manage to link in darshan and MPI statically I had to put in the “lib.a” files by hand as the “darshan-config –pre/post” scripts where still trying to link in darshan dynamically.
>>Here is the link line I used that seem to generate a binary.
>>
>>
>>mpiifort -DLinux -Dx86_64 -DINTEL -DCompLevel16 -DX86_64 -D__x86_64__ -static-intel -static_mpi -DOpenMP -openmp -DSWAP -O3 -g -o /hpcdata/salmr0/mspain/usp2hdf5/dds/bin/Linux/3.0/x86_64/ddswriter.revUnversioned /hpc/tstapps/intel/x86_64/ics-2016.update.1/compilers_and_libraries_2016.1.150/linux/mpi/intel64/lib/libmpifort.a -L/hpc/tstapps/SLES/3.0/x86_64/darshan/3.0.0/lib /hpc/tstapps/SLES/3.0/x86_64/darshan/3.0.0/lib/libdarshan.a -lz -Wl,@/hpc/tstapps/SLES/3.0/x86_64/darshan/3.0.0/share/ld-opts/darshan-base-ld-opts svn_version.o PartitionLayout_mod.o ddswriter.o -L/tstapps/asi/src/dds/lib/Linux/3.0/x86_64/Intel-16.0.1.150MPI5.1.2MKL11.3.1 -ldds_r3 -lgio -lfhost -L/hpc/tstapps/SLES/3.0/x86_64/darshan/3.0.0/lib -Wl,--start-group /hpc/tstapps/SLES/3.0/x86_64/darshan/3.0.0/lib/libdarshan.a /hpc/tstapps/SLES/3.0/x86_64/darshan/3.0.0/lib/libdarshan-stubs.a -Wl,--end-group -lz
>>
>>
>>
>>Here is what the binary look like:
>>
>>hpcbuild02(salmr0)1181:ldd bin/Linux/3.0/x86_64/ddswriter
>> linux-vdso.so.1 => (0x00007ffe791a7000)
>> libz.so.1 => /lib64/libz.so.1 (0x00002aafaab7c000)
>> libdl.so.2 => /lib64/libdl.so.2 (0x00002aafaad92000)
>> librt.so.1 => /lib64/librt.so.1 (0x00002aafaaf96000)
>> libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aafab1a0000)
>> libm.so.6 => /lib64/libm.so.6 (0x00002aafab3bd000)
>> libc.so.6 => /lib64/libc.so.6 (0x00002aafab636000)
>> /lib64/ld-linux-x86-64.so.2 (0x00002aafaa95a000)
>> libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002aafab9b3000)
>>hpcbuild02(salmr0)1182:nm bin/Linux/3.0/x86_64/ddswriter|grep darshan
>>0000000000482e90 T darshan_clean_file_path
>>0000000000483210 t darshan_common_val_compare
>>00000000004830c0 T darshan_common_val_counter
>>00000000004831a0 t darshan_common_val_walker
>>000000000101e860 b darshan_core
>>0000000000463f50 t darshan_core_cleanup
>>00000000004655e0 T darshan_core_initialize
>>0000000000fcfa80 d darshan_core_mutex
>>0000000000464720 T darshan_core_register_module
>>0000000000464b80 T darshan_core_register_record
>>0000000000461c10 T darshan_core_shutdown
>>0000000000464840 T darshan_core_unregister_module
>>0000000000465310 T darshan_core_unregister_record
>>00000000004655b0 T darshan_core_wtime
>>0000000000463e40 t darshan_deflate_buffer..0
>>00000000004642b0 t darshan_get_shared_records
>>00000000004806a0 T darshan_hash
>>0000000000fcfab0 d darshan_mem_alignment
>>0000000000fcfaec d darshan_mem_alignment
>>0000000000b64848 r darshan_module_names
>>0000000000b6487c r darshan_module_versions
>>0000000000479ae0 T darshan_mpiio_shutdown_bench_setup
>>0000000000fcfa20 D darshan_path_exclusions
>>0000000000470250 T darshan_posix_shutdown_bench_setup
>>0000000000461750 T darshan_shutdown_bench
>>0000000000483230 T darshan_variance_reduce
>>0000000000483170 T darshan_walk_common_vals
>>
>>
>>
>>The problem I am having now is that I don’t get a darshan log file when I run my executable. I configured darshan with this configure flag “--with-log-path-by-env=DARSHANLOGPATH”. I set that environment variable in my shell to some directory and then run my executable. The executable runs fine but no darshan log file gets generated.
>>
>>Any ideas? Thanks in advanced.
>>
>>Rene
>>
>>
>>
>>
>>
>>
>>On 5/20/16, 7:33 AM, "darshan-users-bounces at lists.mcs.anl.gov on behalf of Phil Carns" <darshan-users-bounces at lists.mcs.anl.gov on behalf of carns at mcs.anl.gov> wrote:
>>
>>>Hi Rene,
>>>
>>>Does mpiifort support the "-show" option (or something similar) to
>>>display the underlying link command that it is using?
>>>
>>>I can describe the manual steps needed to add Darshan support just to
>>>see if you can get it working, and then we can maybe circle back to why
>>>the Darshan-generated script is triggering dynamic linking. In general,
>>>there are two sets of extra link flags that Darshan needs to add to the
>>>link command. Some should appear before the user-specified libraries in
>>>the true link command, some should appear after them. On my test box
>>>they happen to look like this:
>>>
>>>pcarns at carns-x1:~/working/install/bin$ darshan-config --pre-ld-flags
>>>-L/home/pcarns/working/install/lib -ldarshan -lz
>>>-Wl,@/home/pcarns/working/install/share/ld-opts/darshan-base-ld-opts
>>>
>>>pcarns at carns-x1:~/working/install/bin$ darshan-config --post-ld-flags
>>>-L/home/pcarns/working/install/lib -Wl,--start-group -ldarshan
>>>-ldarshan-stubs -Wl,--end-group -lz -lrt -lpthread
>>>
>>>On most MPI implementations there is also one more library that you have
>>>to add for MPI function wrapping to work. Darshan-config doesn't add
>>>that because it is dependent on the MPI implementation. On most versions
>>>of MPI this is either -lmpifort or -lfmpich. I'm not sure which one
>>>Intel MPI uses now.
>>>
>>>You might be able to do all of that external to mpiifort, but I'm not
>>>sure. If so, then your link command would look something like this
>>>(possibly swapping out -lmpifort for -lfmpich or omitting it entirely):
>>>
>>>mpiifort -DLinux -Dx86_64 -DINTEL -DCompLevel16 -DX86_64 -D__x86_64__ -static-intel -static_mpi -DOpenMP -openmp -DSWAP -O3 -g -o ./ddswriter.revUnversioned -lmpifort `darshan-config --pre-ld-flags` PartitionLayout_mod.o ddswriter.o -L/tstapps/asi/src/dds/lib/Linux/3.0/x86_64/Intel-16.0.1.150MPI5.1.2MKL11.3.1 -ldds_r3 -lgio -lfhost `darshan-config --post-ld-flags`
>>>
>>>
>>>If that doesn't work then you'll need to find out what the underlying
>>>link command looks like (via -show or similar) and insert it there.
>>>
>>>thanks,
>>>-Phil
>>>
>>
>>
>>_______________________________________________
>>Darshan-users mailing list
>>Darshan-users at lists.mcs.anl.gov
>>https://lists.mcs.anl.gov/mailman/listinfo/darshan-users
More information about the Darshan-users
mailing list