[Darshan-commits] [Git][darshan/darshan][master] 5 commits: update regression scripts to use .darshan output
Shane Snyder
xgitlab at cels.anl.gov
Mon Mar 28 11:50:57 CDT 2016
Shane Snyder pushed to branch master at darshan / darshan
Commits:
fd0a9413 by Shane Snyder at 2016-03-28T11:50:21-05:00
update regression scripts to use .darshan output
- - - - -
edb0a0b0 by Shane Snyder at 2016-03-28T11:50:21-05:00
add example output logs for x86_64 & ppc64
- - - - -
6f7229a3 by Shane Snyder at 2016-03-28T11:50:21-05:00
fix makefile clean to remove darshan-diff
- - - - -
a5778368 by Shane Snyder at 2016-03-28T11:50:21-05:00
update documentation for darshan-runtime component
Darshan now provides debugging tips for cases where log files
are not generated.
- - - - -
8f3e3c9e by Shane Snyder at 2016-03-28T11:50:21-05:00
Updated ChangeLog prior to 3.0.0 offiicial release
- - - - -
9 changed files:
- ChangeLog
- darshan-runtime/doc/darshan-runtime.txt
- + darshan-test/example-output/mpi-io-test-ppc64-3.0.0.darshan
- + darshan-test/example-output/mpi-io-test-x86_64-3.0.0.darshan
- darshan-test/regression/test-cases/cxxpi.sh
- darshan-test/regression/test-cases/fperf-f77.sh
- darshan-test/regression/test-cases/fperf-f90.sh
- darshan-test/regression/test-cases/mpi-io-test.sh
- darshan-util/Makefile.in
Changes:
=====================================
ChangeLog
=====================================
--- a/ChangeLog
+++ b/ChangeLog
@@ -4,6 +4,11 @@ Darshan Release Change Log
Darshan-3.0.0
=============
+* update docs to give debugging tips for cases where
+ Darshan logs are not generated
+* fix shared library regression test script to check for
+ potential errors with Darshan symbols rather than
+ failing silently in these cases
* bug fix for determining minimum non-zero counters in
shared file reductions in all modules
* loosen Darshan's PMPI symbol check to prevent inadvertent
=====================================
darshan-runtime/doc/darshan-runtime.txt
=====================================
--- a/darshan-runtime/doc/darshan-runtime.txt
+++ b/darshan-runtime/doc/darshan-runtime.txt
@@ -213,17 +213,36 @@ environment variable to insert instrumentation at run time. The executables
should be compiled using the normal, unmodified MPI compiler.
To use this mechanism, set the `LD_PRELOAD` environment variable to the full
-path to the Darshan shared library, as in this example:
+path to the Darshan shared library. The preferred method of inserting Darshan
+instrumentation in this case is to set the LD_PRELOAD variable specifically
+for the application of interest. Typically this is possible using
+command line arguments offered by the `mpirun` or `mpiexec` scripts or by
+the job scheduler:
----
-export LD_PRELOAD=/home/carns/darshan-install/lib/libdarshan.so
+mpiexec -n 4 -env LD_PRELOAD /home/carns/darshan-install/lib/libdarshan.so mpi-io-test
+----
+
+----
+srun -n 4 --export=LD_PRELOAD=/home/carns/darshan-install/lib/libdarshan.so mpi-io-test
+----
+
+For sequential programs, the following will set LD_PRELOAD for process duration only:
+
----
+env LD_PRELOAD=/home/carns/darshan-install/lib/libdarshan.so mpi-io-test
+----
+
+Other environments may have other specific options for controlling this behavior.
+Please check your local site documentation for details.
-You can then run your application as usual. Some environments may require a
-special `mpirun` or `mpiexec` command line argument to propagate the
-environment variable to all processes. Other environments may require a
-scheduler submission option to control this behavior. Please check your
-local site documentation for details.
+It is also possible to just export LD_PRELOAD as follows, but it is recommended
+against doing that to prevent Darshan and MPI symbols from being pulled into
+unrelated binaries:
+
+----
+export LD_PRELOAD=/home/carns/darshan-install/lib/libdarshan.so
+----
=== Instrumenting dynamically-linked Fortran applications
@@ -511,3 +530,47 @@ behavior at runtime:
* DARSHAN_LOGPATH: specifies the path to write Darshan log files to. Note that this directory needs to be formatted using the darshan-mk-log-dirs script.
* DARSHAN_LOGFILE: specifies the path (directory + Darshan log file name) to write the output Darshan log to. This overrides the default Darshan behavior of automatically generating a log file name and adding it to a log file directory formatted using darshan-mk-log-dirs script.
* DARSHAN_MODMEM: specifies the maximum amount of memory (in MiB) a Darshan instrumentation module can consume at runtime.
+
+== Debugging
+
+=== No log file
+
+In cases where Darshan is not generating a log file for an application, some common things to check are:
+
+* Check stderr to ensure Darshan isn't indicating any internal errors (e.g., invalid log file path)
+
+For statically linked executables:
+
+* Ensure that Darshan symbols are present in the underlying executable by running `nm` on it:
+----
+> nm test | grep darshan
+0000000000772260 b darshan_core
+0000000000404440 t darshan_core_cleanup
+00000000004049b0 T darshan_core_initialize
+000000000076b660 d darshan_core_mutex
+00000000004070a0 T darshan_core_register_module
+----
+
+* Make sure the application executable is statically linked:
+ ** In general, we encourage the use of purely statically linked executables when using the static
+instrumentation method given in link:darshan-runtime.html#_instrumenting_statically_linked_applications[Section 5]
+ ** If purely static executables are not an option, we encourage users to use the LD_PRELOAD method of
+instrumentation given in link:darshan-runtime.html#_instrumenting_dynamically_linked_applications[Section 6]
+ ** Statically linked executables are the default on Cray platforms and the IBM BG platforms;
+statically linked executables can be explicitly requested using the `-static` compile option to most compilers
+ ** You can verify that an executable is purely statically linked by using the `file` command:
+----
+> file mpi-io-test
+mpi-io-test: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, for GNU/Linux 2.6.24, BuildID[sha1]=9893e599e7a560159ccf547b4c4ba5671f65ba32, not stripped
+----
+
+* Ensure that the linker is correctly linking in Darshan's runtime libraries:
+ ** A common mistake is to explicitly link in the underlying MPI libraries (e.g., `-lmpich` or `-lmpichf90`)
+in the link command, which can interfere with Darshan's instrumentation
+ *** These libraries are usually linked in automatically by the compiler
+ *** MPICH's `mpicc` comipler's `-show` flag can be used to examine the invoked link command, for instance
+ ** The linker's `-y` option can be used to verify that Darshan is properly intercepting MPI_Init
+function (e.g. by setting `CFLAGS='-Wl,-yMPI_Init'`), which it uses to initialize its runtime structures
+----
+/usr/common/software/darshan/3.0.0-pre3/lib/libdarshan.a(darshan-core-init-finalize.o): definition of MPI_Init
+----
=====================================
darshan-test/example-output/mpi-io-test-ppc64-3.0.0.darshan
=====================================
Binary files /dev/null and b/darshan-test/example-output/mpi-io-test-ppc64-3.0.0.darshan differ
=====================================
darshan-test/example-output/mpi-io-test-x86_64-3.0.0.darshan
=====================================
Binary files /dev/null and b/darshan-test/example-output/mpi-io-test-x86_64-3.0.0.darshan differ
=====================================
darshan-test/regression/test-cases/cxxpi.sh
=====================================
--- a/darshan-test/regression/test-cases/cxxpi.sh
+++ b/darshan-test/regression/test-cases/cxxpi.sh
@@ -3,7 +3,7 @@
PROG=cxxpi
# set log file path; remove previous log if present
-export DARSHAN_LOGFILE=$DARSHAN_TMP/${PROG}.darshan.gz
+export DARSHAN_LOGFILE=$DARSHAN_TMP/${PROG}.darshan
rm -f ${DARSHAN_LOGFILE}
# compile
=====================================
darshan-test/regression/test-cases/fperf-f77.sh
=====================================
--- a/darshan-test/regression/test-cases/fperf-f77.sh
+++ b/darshan-test/regression/test-cases/fperf-f77.sh
@@ -3,7 +3,7 @@
PROG=fperf-f77
# set log file path; remove previous log if present
-export DARSHAN_LOGFILE=$DARSHAN_TMP/${PROG}.darshan.gz
+export DARSHAN_LOGFILE=$DARSHAN_TMP/${PROG}.darshan
rm -f ${DARSHAN_LOGFILE}
# compile
=====================================
darshan-test/regression/test-cases/fperf-f90.sh
=====================================
--- a/darshan-test/regression/test-cases/fperf-f90.sh
+++ b/darshan-test/regression/test-cases/fperf-f90.sh
@@ -3,7 +3,7 @@
PROG=fperf-f90
# set log file path; remove previous log if present
-export DARSHAN_LOGFILE=$DARSHAN_TMP/${PROG}.darshan.gz
+export DARSHAN_LOGFILE=$DARSHAN_TMP/${PROG}.darshan
rm -f ${DARSHAN_LOGFILE}
# compile
=====================================
darshan-test/regression/test-cases/mpi-io-test.sh
=====================================
--- a/darshan-test/regression/test-cases/mpi-io-test.sh
+++ b/darshan-test/regression/test-cases/mpi-io-test.sh
@@ -3,7 +3,7 @@
PROG=mpi-io-test
# set log file path; remove previous log if present
-export DARSHAN_LOGFILE=$DARSHAN_TMP/${PROG}.darshan.gz
+export DARSHAN_LOGFILE=$DARSHAN_TMP/${PROG}.darshan
rm -f ${DARSHAN_LOGFILE}
# compile
=====================================
darshan-util/Makefile.in
=====================================
--- a/darshan-util/Makefile.in
+++ b/darshan-util/Makefile.in
@@ -153,7 +153,7 @@ endif
clean::
- rm -f *.o *.po *.a darshan-analyzer darshan-convert darshan-parser jenkins-hash-gen
+ rm -f *.o *.po *.a darshan-analyzer darshan-convert darshan-diff darshan-parser jenkins-hash-gen
distclean:: clean
rm -f darshan-runtime-config.h aclocal.m4 autom4te.cache/* config.status config.log Makefile util/bin/darshan-job-summary.pl
View it on GitLab: https://xgitlab.cels.anl.gov/darshan/darshan/compare/63512a55b90b9a9fbc7d4b184cb2e6f30d8fe608...8f3e3c9e371dbdac32fca29392eb1b22766c6f36
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/darshan-commits/attachments/20160328/61000729/attachment.html>
More information about the Darshan-commits
mailing list