[petsc-dev] Apparent bug with PGI on Titan with each thread calling a serial LU solve

[Barry Smith]

  Yup, doesn’t work in debug mode, this is a documented feature.  Because debug mode has lots more global variable like things.

   Barry

On May 8, 2014, at 3:18 PM, Mark Adams <mfadams at lbl.gov> wrote:

> Barry, the test code works in OPT but in debug mode it gets this error.
> Thanks,
> Mark
> 
> 
> 
> On Wed, May 7, 2014 at 1:02 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
> 
>    Mark,
> 
>    Are you sure the PGI version is built with the correct branch of barry/make-petscoptionsobject-nonglobal ?
> 
>     In that branch line 516 of aoptions.c has         case OPTION_INT:   not a PetscFree
> 
>   Meanwhile in master line 516 has   ierr = PetscFree(PetscOptionsObject.title);CHKERRQ(ierr);
> 
> 
>   Barry
> 
> 
> On May 7, 2014, at 2:47 PM, Mark Adams <mfadams at lbl.gov> wrote:
> 
> > Barry, made the LU solver "thread safe" in that each thread calls a serial solve, for Ed (cc'ed).
> >
> > Ed has a test code (attached) that can be run with 16 threads (aprun -n 1 -d 16) and NUM_OPENMP_THREADS=16.  This code seems to work with Intel compilers but fails with PGI.
> >
> > I've appended a stack trace.  Any ideas?
> >
> > Mark
> > Note, the line numbers are not quite right in tpetsc.F90 but the rest look OK.
> >
> > #11 tpetsc () at /autofs/na3_home1/adams/petsc/src/ksp/ksp/examples/tutorials/tpetsc.F90:144 (at 0x0000000000422afa)
> > #10 matcreateseqaij_ () at /autofs/na3_home1/adams/petsc/src/mat/impls/aij/seq/ftn-custom/zaijf.c:14 (at 0x0000000000474e95)
> > #9 MatCreateSeqAIJ () at /autofs/na3_home1/adams/petsc/src/mat/impls/aij/seq/aij.c:3574 (at 0x000000000053c8b1)
> > #8 MatSetType () at /autofs/na3_home1/adams/petsc/src/mat/interface/matreg.c:71 (at 0x000000000050b292)
> > #7 MatCreate_SeqAIJ () at /autofs/na3_home1/adams/petsc/src/mat/impls/aij/seq/aij.c:4129 (at 0x000000000053f24a)
> > #6 MatCreate_SeqAIJ_Inode () at /autofs/na3_home1/adams/petsc/src/mat/impls/aij/seq/inode2.c:99 (at 0x000000000058b205)
> > #5 PetscOptionsEnd_Private () at /autofs/na3_home1/adams/petsc/src/sys/objects/aoptions.c:516 (at 0x000000000043fc25)
> > #4 PetscFreeAlign () at /autofs/na3_home1/adams/petsc/src/sys/memory/mal.c:72 (at 0x0000000000483f89)
> > #3 free () from /dsl/lib64/libc-2.11.3.so (at 0x00002aaab5bcc4fc)
> > #2 malloc_printerr () from /dsl/lib64/libc-2.11.3.so (at 0x00002aaab5bc7558)
> > #1 __libc_message () from /dsl/lib64/libc-2.11.3.so (at 0x00002aaab5bc1e2f)
> > #0 abort () from /dsl/lib64/libc-2.11.3.so (at 0x00002aaab5b85fb0)
> >
> >
> >
> > <tpetsc.F90>
> 
> 
> <error_in_debug_mode>