[petsc-users] Petsc cannot be initialized on vesta in some --mode options

[Jed Brown]

Roc Wang <pengxwang at hotmail.com> writes:
> The configure.log is attached.
> I also tried running the program with command like:
> qsub -n <number of nodes> -t 10 --mode <ranks per node> --env "F00=a:BAR=b:PAMID_COLLECTIVES=0" ./x.r 
>
> For 1024 as the total number of ranks, the program was able to run in
> c1, c16, c32, and c64 mode, but still petsc cannot be initialized in
> c2, c4, and c8 mode. The runtime log files for c8 mode were
> attached. Thanks.

Hmm, appears to be this line erroring.

      ierr     = MPI_Bcast(&len,1,MPIU_SIZE_T,0,PETSC_COMM_WORLD);CHKERRQ(ierr);

But at least some processes are stuck in an earlier
PetscOptionsInsertFile (which contains MPI_Bcast).  This is like the
problem we have seen before, but it used to go away with
PAMID_COLLECTIVES=0.  Can you run with -skip_petscrc?  That will skip a
couple MPI_Bcasts and might get it working again.


Jeff, has the Vesta stack changed recently?  Any reason why
PAMID_COLLECTIVES=0 would not be taking effect here?

I'll try to reproduce Roc's findings later today.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20140122/5ed5e432/attachment.pgp>