[Darshan-commits] [Git][darshan/darshan][master] 15 commits: remove old bluegene regression test stuff
Shane Snyder
xgitlab at cels.anl.gov
Mon Jul 13 20:35:19 CDT 2020
Shane Snyder pushed to branch master at darshan / darshan
Commits:
29137c36 by Shane Snyder at 2020-04-24T12:47:42-05:00
remove old bluegene regression test stuff
- - - - -
7f395272 by Shane Snyder at 2020-04-24T12:47:42-05:00
add static/dynamic variants of prof-conf test
- - - - -
2c3ac243 by Shane Snyder at 2020-04-24T12:47:42-05:00
rename workstation-static test to cc-wrapper test
- - - - -
53d59541 by Shane Snyder at 2020-04-24T12:47:42-05:00
rename dynamic test case to ld-preload
- - - - -
27eb5aad by Shane Snyder at 2020-04-24T12:47:42-05:00
updated regression readme for new test cases
- - - - -
c8c7470f by Shane Snyder at 2020-04-24T12:47:42-05:00
typo
- - - - -
59bb8f1d by Shane Snyder at 2020-04-24T12:47:42-05:00
try static/dynamic theta tests for gitlab ci
- - - - -
6ce8bd7f by Shane Snyder at 2020-04-24T12:51:18-05:00
static test case not setting link_type properly
- - - - -
b9420c4b by Shane Snyder at 2020-04-26T22:29:55-05:00
attempt to split test into static/dynamic jobs
- - - - -
ccbd97bb by Shane Snyder at 2020-04-27T09:11:54-05:00
more gitlab ci debugging..
- - - - -
61594841 by Shane Snyder at 2020-04-27T09:54:11-05:00
restrict number of test cases for now
- - - - -
5258fc84 by Shane Snyder at 2020-04-27T10:16:29-05:00
allow project members to update ci jobs
- - - - -
86fcfce7 by Shane Snyder at 2020-04-27T10:52:49-05:00
re-enable both static/dynamic test cases
- - - - -
34ec51db by Shane Snyder at 2020-07-13T16:37:26-05:00
run all test cases instead of just 1
- - - - -
ba3aa877 by Shane Snyder at 2020-07-13T20:35:16-05:00
Merge branch 'dev-new-regression-tests' into 'master'
new regression test cases
See merge request darshan/darshan!48
- - - - -
9 changed files:
- .gitlab-ci.yml
- − darshan-test/regression/README.BG.ALCF.txt
- darshan-test/regression/README.txt
- − darshan-test/regression/bg-profile-conf-alcf/runjob.sh
- darshan-test/regression/cray-module-alcf/runjob.sh
- darshan-test/regression/workstation-static/env.sh → darshan-test/regression/workstation-cc-wrapper/env.sh
- darshan-test/regression/workstation-dynamic/env.sh → darshan-test/regression/workstation-ld-preload/env.sh
- darshan-test/regression/workstation-profile-conf/env.sh → darshan-test/regression/workstation-profile-conf-dynamic/env.sh
- darshan-test/regression/bg-profile-conf-alcf/env.sh → darshan-test/regression/workstation-profile-conf-static/env.sh
Changes:
=====================================
.gitlab-ci.yml
=====================================
@@ -29,7 +29,7 @@ build_darshan:
paths:
- install/
-test_darshan:
+test_darshan_static:
tags:
- shell
- ecp-theta
@@ -43,8 +43,7 @@ test_darshan:
stage: test
script:
- - ls $PWD/install
- - ls $PWD/install/bin
+ - export CRAYPE_LINK_TYPE=static
- darshan-test/regression/run-all.sh $PWD/install $PWD/scratch cray-module-alcf
artifacts:
@@ -53,3 +52,34 @@ test_darshan:
- $PWD/scratch/*.debuglog
- $PWD/scratch/*.out
- $PWD/scratch/*.err
+
+ resource_group: debug-queue
+
+test_darshan_dynamic:
+ tags:
+ - shell
+ - ecp-theta
+
+ rules:
+ - if: '$CI_PIPELINE_SOURCE == "web" && $RUN_TESTS == "true"'
+ when: on_success
+ - if: '$CI_PIPELINE_SOURCE == "schedules" && $RUN_TESTS == "true"'
+ when: on_success
+
+ stage: test
+
+ script:
+ - export CRAYPE_LINK_TYPE=dynamic
+ - darshan-test/regression/run-all.sh $PWD/install $PWD/scratch cray-module-alcf
+ - ldd $PWD/scratch/mpi-io-test
+ - nm $PWD/scratch/mpi-io-test | grep darshan
+ - nm $PWD/scratch/mpi-io-test | grep MPI
+
+ artifacts:
+ paths:
+ - $PWD/scratch/*.darshan
+ - $PWD/scratch/*.debuglog
+ - $PWD/scratch/*.out
+ - $PWD/scratch/*.err
+
+ resource_group: debug-queue
=====================================
darshan-test/regression/README.BG.ALCF.txt deleted
=====================================
@@ -1,29 +0,0 @@
-See README.txt for general instructions. This file contains notes for testing on the Blue Gene platform
-(more specifically: cetus.alcf.anl.gov). This example assumes that you are using the MPICH profile conf
-method to add instrumentation.
-
-To run regression tests:
-
-- compile and install both darshan-runtime and darshan-util in the same directory
- examples:
-
- # darshan runtime
- ../configure --with-mem-align=16 --with-log-path=/projects/SSSPPg/carns/darshan-logs --prefix=/home/carns/working/darshan/install-cetus --with-jobid-env=COBALT_JOBID --with-zlib=/soft/libraries/alcf/current/gcc/ZLIB --host=powerpc-bgp-linux CC=/bgsys/drivers/V1R2M2/ppc64/comm/bin/gcc/mpicc
- make install
-
- # darshan util
- ../configure --prefix=/home/carns/working/darshan/install-cetus
- make install
-
-- start a screen session by running "screen"
- note: this is suggested because the tests may take a while to complete depending on scheduler
- availability
-
-- within the screen session, set your path to point to a stock set of MPI compiler scripts
- export PATH=/bgsys/drivers/V1R2M2/ppc64/comm/bin/gcc:$PATH
-
-- run regression tests
- ./run-all.sh /home/carns/working/darshan/install-cetus /projects/SSSPPg/carns/darshan-test bg-profile-conf-alcf
- note: the f90 test is expected to fail due to a known problem in the profiling interface for the
- F90 MPICH implementation on Mira.
-
=====================================
darshan-test/regression/README.txt
=====================================
@@ -8,14 +8,18 @@ The master script must be executed with three arguments:
2) path to temporary directory (for building executables, collecting logs,
etc. during test)
3) platform type; options include:
- - workstation-static (for static instrumentation on a standard workstation)
- - workstation-dynamic (for dynamic instrumentation on a standard workstation)
- - workstation-profile-conf (for static instrumentation using MPI profiling
- configuration hooks on a standard workstation)
- - bg-profile-conf-alcf (for static instrumentation using MPI profiling configuration
- hooks on BGQ platforms @ the ALCF only)
- - cray-module-alcf (for static instrumentation using a Darshan Cray module on
- Cray systems @ the ALCF only)
+ - workstation-cc-wrapper (for static/dynamic instrumentation on a standard
+ workstation using Darshan compiler wrappers)
+ - workstation-profile-conf-static (for static instrumentation using MPI
+ profiling configuration hooks on a standard workstation)
+ - workstation-profile-conf-dynamic (for dynamic instrumentation using MPI
+ profiling configuration hooks on a standard workstation)
+ - workstation-ld-preload (for dynamic instrumentation via LD_PRELOAD on a
+ standard workstation)
+ - cray-module-alcf (for static/dynamic instrumentation using a Darshan
+ Cray module on Cray systems @ the ALCF only)
+ - cray-module-nersc (for static/dynamic instrumentation using a Darshan
+ Cray module on Cray systems @ NERSC only)
The platform type should map to a subdirectory containing scripts
that describe how to perform platform-specific tasks (like loading or
=====================================
darshan-test/regression/bg-profile-conf-alcf/runjob.sh deleted
=====================================
@@ -1,31 +0,0 @@
-#!/bin/bash
-
-# convert DXT env setting
-if [ -n "${DXT_ENABLE_IO_TRACE+defined}" ]; then
- DXT_ENV="--env DXT_ENABLE_IO_TRACE=$DXT_ENABLE_IO_TRACE"
-fi
-
-# submit job and get job id
-jobid=`qsub --env DARSHAN_LOGFILE=$DARSHAN_LOGFILE $DXT_ENV --mode c16 --proccount $DARSHAN_DEFAULT_NPROCS -A radix-io -t 10 -n 1 --output $DARSHAN_TMP/$$-tmp.out --error $DARSHAN_TMP/$$-tmp.err --debuglog $DARSHAN_TMP/$$-tmp.debuglog "$@"`
-if [ $? -ne 0 ]; then
- echo "Error: failed to qsub $@"
- exit 1
-fi
-
-output="foo"
-rc=0
-
-# loop as long as qstat succeeds and shows information about job
-while [ -n "$output" -a "$rc" -eq 0 ]; do
- sleep 5
- output=`qstat $jobid`
- rc=$?
-done
-
-# look for return code
-grep "exit code of 0" $DARSHAN_TMP/$$-tmp.debuglog >& /dev/null
-if [ $? -ne 0 ]; then
- exit 1
-else
- exit 0
-fi
=====================================
darshan-test/regression/cray-module-alcf/runjob.sh
=====================================
@@ -6,7 +6,7 @@ if [ -n "${DXT_ENABLE_IO_TRACE+defined}" ]; then
fi
# submit job and get job id
-jobid=`qsub --env DARSHAN_LOGFILE=$DARSHAN_LOGFILE --env DARSHAN_DEFAULT_NPROCS=$DARSHAN_DEFAULT_NPROCS $DXT_ENV --proccount $DARSHAN_DEFAULT_NPROCS -A CSC250STDM12 -q debug-cache-quad -t 20 -n 1 --output $DARSHAN_TMP/$$-tmp.out --error $DARSHAN_TMP/$$-tmp.err --debuglog $DARSHAN_TMP/$$-tmp.debuglog $DARSHAN_TESTDIR/$DARSHAN_PLATFORM/cobalt-submit.sh "$@"`
+jobid=`qsub --env DARSHAN_LOGFILE=$DARSHAN_LOGFILE --env DARSHAN_DEFAULT_NPROCS=$DARSHAN_DEFAULT_NPROCS $DXT_ENV --proccount $DARSHAN_DEFAULT_NPROCS -A CSC250STDM12 -q debug-cache-quad -t 20 -n 1 --run_project --output $DARSHAN_TMP/$$-tmp.out --error $DARSHAN_TMP/$$-tmp.err --debuglog $DARSHAN_TMP/$$-tmp.debuglog $DARSHAN_TESTDIR/$DARSHAN_PLATFORM/cobalt-submit.sh "$@"`
if [ $? -ne 0 ]; then
echo "Error: failed to qsub $@"
=====================================
darshan-test/regression/workstation-static/env.sh → darshan-test/regression/workstation-cc-wrapper/env.sh
=====================================
@@ -16,7 +16,7 @@
# variables (as in a dynamically linked environment), or generate mpicc
# wrappers (as in a statically linked environment).
-# Notes specific to this platform (workstation-static)
+# Notes specific to this platform (workstation-cc-wrapper)
########################
# This particular env script assumes that mpicc and its variants for other
# languages are already in the path. The compiler scripts to be used in
=====================================
darshan-test/regression/workstation-dynamic/env.sh → darshan-test/regression/workstation-ld-preload/env.sh
=====================================
@@ -16,11 +16,11 @@
# variables (as in a dynamically linked environment), or generate mpicc
# wrappers (as in a statically linked environment).
-# Notes specific to this platform (workstation-dynamic)_
+# Notes specific to this platform (workstation-ld-preload)
########################
# This particular env script assumes that mpicc and its variants for other
# languages are already in the path, and that they will produce dynamic
-# executables by default. Test programs are compile usign the existing
+# executables by default. Test programs are compiled using the existing
# scripts, and LD_PRELOAD is set to enable instrumentation.
# The runjob command is just mpiexec, no scheduler
=====================================
darshan-test/regression/workstation-profile-conf/env.sh → darshan-test/regression/workstation-profile-conf-dynamic/env.sh
=====================================
@@ -16,10 +16,10 @@
# variables (as in a dynamically linked environment), or generate mpicc
# wrappers (as in a statically linked environment).
-# Notes specific to this platform (workstation-dynamic)_
+# Notes specific to this platform (workstation-profile-conf-dynamic)
########################
# This particular env script assumes that mpicc and its variants for other
-# languages are already in the path, and that they will produce static
+# languages are already in the path, and that they will produce dynamic
# executables by default. Darshan instrumentation is added by specifying
# a profiling configuration file using environment variables.
=====================================
darshan-test/regression/bg-profile-conf-alcf/env.sh → darshan-test/regression/workstation-profile-conf-static/env.sh
=====================================
@@ -16,24 +16,26 @@
# variables (as in a dynamically linked environment), or generate mpicc
# wrappers (as in a statically linked environment).
-# Notes specific to this platform (bg-profile-conf-alcf)
+# Notes specific to this platform (workstation-profile-conf-static)
########################
# This particular env script assumes that mpicc and its variants for other
# languages are already in the path, and that they will produce static
# executables by default. Darshan instrumentation is added by specifying
# a profiling configuration file using environment variables.
-# the RUNJOB command is the most complex part here. We use a script that submits
-# a cobalt job, waits for its completion, and checks its return status
+# The runjob command is just mpiexec, no scheduler
export DARSHAN_CC=mpicc
export DARSHAN_CXX=mpicxx
export DARSHAN_F77=mpif77
export DARSHAN_F90=mpif90
-export MPICC_PROFILE=$DARSHAN_PATH/share/mpi-profile/darshan-bg-cc
-export MPICXX_PROFILE=$DARSHAN_PATH/share/mpi-profile/darshan-bg-cxx
-export MPIF90_PROFILE=$DARSHAN_PATH/share/mpi-profile/darshan-bg-f
-export MPIF77_PROFILE=$DARSHAN_PATH/share/mpi-profile/darshan-bg-f
+export MPICC_PROFILE=$DARSHAN_PATH/share/mpi-profile/darshan-cc-static
+export MPICXX_PROFILE=$DARSHAN_PATH/share/mpi-profile/darshan-cxx-static
+export MPIF90_PROFILE=$DARSHAN_PATH/share/mpi-profile/darshan-f-static
+export MPIF77_PROFILE=$DARSHAN_PATH/share/mpi-profile/darshan-f-static
+# MPICH 3.1.1 and newer use MPIFORT rather than MPIF90 and MPIF77 in env var
+# name
+export MPIFORT_PROFILE=$DARSHAN_PATH/share/mpi-profile/darshan-f-static
-export DARSHAN_RUNJOB="bg-profile-conf-alcf/runjob.sh"
+export DARSHAN_RUNJOB="mpiexec -n $DARSHAN_DEFAULT_NPROCS"
View it on GitLab: https://xgitlab.cels.anl.gov/darshan/darshan/compare/8f42b7b241d01d22e95472c3015594d7584baece...ba3aa87711921cb3cf0ad3d0d8d9e8f36108c7da
--
View it on GitLab: https://xgitlab.cels.anl.gov/darshan/darshan/compare/8f42b7b241d01d22e95472c3015594d7584baece...ba3aa87711921cb3cf0ad3d0d8d9e8f36108c7da
You're receiving this email because of your account on xgitlab.cels.anl.gov.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/darshan-commits/attachments/20200713/43ffd631/attachment-0001.html>
More information about the Darshan-commits
mailing list