<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Sep 25, 2019 at 8:40 PM Balay, Satish <<a href="mailto:balay@mcs.anl.gov">balay@mcs.anl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">> Unable to run jsrun -g 1 with option "-n 1"<br>
> Error: It is only possible to use js commands within a job allocation<br>
> unless CSM is running<br>
<br>
<br>
Nope this is a different error message.<br>
<br>
The message suggests - you can't run 'jsrun -g 1 -n 1 binary' Can you try this manually and see<br>
what you get?<br>
<br>
jsrun -g 1 -n 1 printenv<br></blockquote><div><br></div><div>I tested this earlier today and originally when I was figuring out the/a minimal run command:</div><div><br></div><div>22:08 /gpfs/alpine/geo127/scratch/adams$ jsrun -g 1 -n 1 printenv<br>GIT_PS1_SHOWDIRTYSTATE=1<br>XDG_SESSION_ID=494<br>SHELL=/bin/bash<br>HISTSIZE=100<br>PETSC_ARCH=arch-summit-opt64-pgi-cuda<br>SSH_CLIENT=160.91.202.152 48626 22<br>LC_ALL=<br>USER=adams<br></div><div> ...</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Satish<br>
<br>
<br>
On Wed, 25 Sep 2019, Mark Adams via petsc-dev wrote:<br>
<br>
> On Wed, Sep 25, 2019 at 6:23 PM Balay, Satish <<a href="mailto:balay@mcs.anl.gov" target="_blank">balay@mcs.anl.gov</a>> wrote:<br>
> <br>
> > > 18:16 (cb53a04...) ~/petsc-karl$<br>
> ><br>
> > So this is the commit I recommended you test against - and that's what<br>
> > you have got now. Please go ahead and test.<br>
> ><br>
> ><br>
> I sent the log for this. This is the output:<br>
> <br>
> 18:16 (cb53a04...) ~/petsc-karl$ ../arch-summit-opt64idx-pgi-cuda.py<br>
> PETSC_DIR=$PWD<br>
> ===============================================================================<br>
> Configuring PETSc to compile on your system<br>
> <br>
> ===============================================================================<br>
> ===============================================================================<br>
> <br>
> ***** WARNING: F77 (set to<br>
> /autofs/nccs-svm1_sw/summit/.swci/0-core/opt/spack/20180914/linux-rhel7-ppc64le/gcc-4.8.5/pgi-19.4-6acz4xyqjlpoaonjiiqjme2aknrfnzoy/linux<br>
> use ./configure F77=$F77 if you really want to use that value ******<br>
> <br>
> <br>
> ===============================================================================<br>
> <br>
> <br>
> ===============================================================================<br>
> <br>
> ***** WARNING: Using default optimization C flags -O<br>
> <br>
> You might consider manually setting<br>
> optimal optimization flags for your system with<br>
> <br>
> COPTFLAGS="optimization flags" see config/examples/arch-*-opt.py for<br>
> examples<br>
> <br>
> ===============================================================================<br>
> <br>
> <br>
> ===============================================================================<br>
> <br>
> ***** WARNING: You have an older version of Gnu make,<br>
> it will work,<br>
> but may not support all the<br>
> parallel testing options. You can install the<br>
> latest<br>
> Gnu make with your package manager, such as brew or macports, or use<br>
> <br>
> the --download-make option to get the latest Gnu make warning<br>
> message *****<br>
> <br>
> ===============================================================================<br>
> <br>
> TESTING: configureMPIEXEC from<br>
> config.packages.MPI(config/BuildSystem/config/packages/MPI.py:174)<br>
> <br>
> *******************************************************************************<br>
> UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for<br>
> details):<br>
> -------------------------------------------------------------------------------<br>
> Unable to run jsrun -g 1 with option "-n 1"<br>
> Error: It is only possible to use js commands within a job allocation<br>
> unless CSM is running<br>
> 09-25-2019 18:20:13:224 108023 main: Error initializing RM connection.<br>
> Exiting.<br>
> *******************************************************************************<br>
> <br>
> 18:20 1 (cb53a04...) ~/petsc-karl$<br>
> <br>
> <br>
> > [note: the branch is rebased - so 'git pull' won't work -(as you can<br>
> > see from the "(forced update)" message - and '<>' status from git<br>
> > prompt on balay/fix-mpiexec-shell-escape). So perhaps its easier to<br>
> > deal with in detached mode - which makes this obvious]<br>
> ><br>
> <br>
> I got this <> and "fixed" it by deleting the branch and repulling it. I<br>
> guess I needed to fetch also.<br>
> <br>
> Mark<br>
> <br>
> <br>
> ><br>
> > Satish<br>
> ><br>
> ><br>
> > On Wed, 25 Sep 2019, Mark Adams via petsc-dev wrote:<br>
> ><br>
> > > I will test this now but ....<br>
> > ><br>
> > > 17:52 balay/fix-mpiexec-shell-escape= ~/petsc-karl$ git fetch<br>
> > > remote: Enumerating objects: 119, done.<br>
> > > remote: Counting objects: 100% (119/119), done.<br>
> > > remote: Compressing objects: 100% (91/91), done.<br>
> > > remote: Total 119 (delta 49), reused 74 (delta 28)<br>
> > > Receiving objects: 100% (119/119), 132.88 KiB | 0 bytes/s, done.<br>
> > > Resolving deltas: 100% (49/49), completed with 1 local objects.<br>
> > > >From <a href="https://gitlab.com/petsc/petsc" rel="noreferrer" target="_blank">https://gitlab.com/petsc/petsc</a><br>
> > > + b5e99a5...cb53a04 balay/fix-mpiexec-shell-escape -><br>
> > > origin/balay/fix-mpiexec-shell-escape (forced update)<br>
> > > + ffdc635...7eeb5f9 jczhang/feature-sf-on-gpu -><br>
> > > origin/jczhang/feature-sf-on-gpu (forced update)<br>
> > > cb9de97..f9ff08a jolivet/fix-error-col-row -><br>
> > > origin/jolivet/fix-error-col-row<br>
> > > 40ea605..de5ad60 oanam/jacobf/cell-to-ref-mapping -><br>
> > > origin/oanam/jacobf/cell-to-ref-mapping<br>
> > > + ecac953...9fb579e stefanozampini/hypre-cuda-rebased -><br>
> > > origin/stefanozampini/hypre-cuda-rebased (forced update)<br>
> > > 18:16 balay/fix-mpiexec-shell-escape<> ~/petsc-karl$ git checkout<br>
> > > origin/balay/fix-mpiexec-shell-escape<br>
> > > Note: checking out 'origin/balay/fix-mpiexec-shell-escape'.<br>
> > ><br>
> > > You are in 'detached HEAD' state. You can look around, make experimental<br>
> > > changes and commit them, and you can discard any commits you make in this<br>
> > > state without impacting any branches by performing another checkout.<br>
> > ><br>
> > > If you want to create a new branch to retain commits you create, you may<br>
> > > do so (now or later) by using -b with the checkout command again.<br>
> > Example:<br>
> > ><br>
> > > git checkout -b new_branch_name<br>
> > ><br>
> > > HEAD is now at cb53a04... mpiexec: fix shell escape of path-to-mpiexec<br>
> > only<br>
> > > when using autodected-path. Also spectrum MPI uses OMPI_MAJOR_VERSION<br>
> > etc -<br>
> > > so check if mpiexec supports --oversubscribe - before using it.<br>
> > > 18:16 (cb53a04...) ~/petsc-karl$<br>
> > ><br>
> > > On Wed, Sep 25, 2019 at 5:58 PM Balay, Satish <<a href="mailto:balay@mcs.anl.gov" target="_blank">balay@mcs.anl.gov</a>> wrote:<br>
> > ><br>
> > > > Defined "VERSION_GIT" to ""v3.11.3-2242-gb5e99a5""<br>
> > > ><br>
> > > > This is not the latest state - It should be:<br>
> > > ><br>
> > > > commit cb53a042369fb946804f53931a88b58e10588da1 (HEAD -><br>
> > > > balay/fix-mpiexec-shell-escape, origin/balay/fix-mpiexec-shell-escape)<br>
> > > ><br>
> > > > Try:<br>
> > > ><br>
> > > > git fetch<br>
> > > > git checkout origin/balay/fix-mpiexec-shell-escape<br>
> > > ><br>
> > > > Satish<br>
> > > ><br>
> > > > On Wed, 25 Sep 2019, Mark Adams via petsc-dev wrote:<br>
> > > ><br>
> > > > > On Wed, Sep 25, 2019 at 4:57 PM Balay, Satish <<a href="mailto:balay@mcs.anl.gov" target="_blank">balay@mcs.anl.gov</a>><br>
> > wrote:<br>
> > > > ><br>
> > > > > > On Wed, 25 Sep 2019, Mark Adams via petsc-dev wrote:<br>
> > > > > ><br>
> > > > > > > I did test this and sent the log (error).<br>
> > > > > ><br>
> > > > > > Mark,<br>
> > > > > ><br>
> > > > > > I made more changes - can you retry again - and resend log.<br>
> > > > > ><br>
> > > > > > Satish<br>
> > > > > ><br>
> > > > ><br>
> > > ><br>
> > > ><br>
> > ><br>
> ><br>
> ><br>
> <br>
<br>
</blockquote></div></div>