[petsc-dev] Fwd: no petsc on Edison

[Kong, Fande]

On Wed, Jan 11, 2017 at 1:55 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:

>
> > On Jan 11, 2017, at 2:41 PM, Kong, Fande <fande.kong at inl.gov> wrote:
> >
> > We do not run the code with Satish's fixes, instead, we are using
> petsc-3.7.3 and there are not such issues. I guess the issues are
> introduced recently.
>
>   Ok, then I am confused. Maybe master fixes the problem?
>

Barry, I am sorry for confusing you. Let me rephrase a little.

We encountered this issue when we were using PETSc-3.7.4, and we reported
the issue in another thread. Then Satish tried to fix the issue by adding
more entities to the lookup, but we did not try the Satish's branch with
this fix. We are back to use an old release PETSc-3.7.3. PETSc-3.7.3 is
working just fine.

In PETSc-3.7.3, there is no such a function called
PetscTableCreateHashSize. I hence guess the issue was introduced recently.

Fande,




>
>
> >
> > Fande,
> >
> > On Wed, Jan 11, 2017 at 1:31 PM, Mark Adams <mfadams at lbl.gov> wrote:
> > Argh, yes that has to be the local size or a least not the global size.
> >
> > On Wed, Jan 11, 2017 at 3:15 PM, Satish Balay <balay at mcs.anl.gov> wrote:
> > >>>
> > aN=A->cmap->N
> > ierr = PetscTableCreate(aN,aN,&ta);CHKERRQ(ierr);
> > <<<
> >
> > Looks like thats a 'global' size. That doesn't look right.
> >
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__
> bitbucket.org_petsc_petsc_commits_c373ccc6-23chg-2Dsrc_mat_impls_aij_mpi_
> mpimatmatmult.c&d=DQIFAg&c=54IZrppPQZKX9mLzcGdPfFD1hxrcB_
> _aEkJFOKJFd00&r=DUUt3SRGI0_JgtNaS3udV68GRkgV4ts7XKfj2opmiCY&m=QrWHgIE8D6e-
> Soe4EmaMtPH316RnhxnYdz3z8-0BzAI&s=99aU-Suv1V6kVMrH-
> cklX3-0MhHK8doL0qHKG6SD3dg&e=
> >
> > I see Hong had previously updated this code. Will have to recheck..
> >
> > Satish
> >
> > On Wed, 11 Jan 2017, Mark Adams wrote:
> >
> > > >
> > > >
> > > > Must be the same place Fande encountered this error.
> > > >
> > > >
> > > Satish, here is our stack trace. Do you have any thoughts vis-a-vis
> Fande's
> > > experience what the problem is here?  My best guess is that the local
> > > matrix size is huge, but this code can have some very large stencils.
> > >
> > > Treb: about how many processors have this error? Is is it just one, a
> few,
> > > or tons?
> > >
> > > [5]PETSC ERROR: Argument out of range
> > > [5]PETSC ERROR: A really huge hash is being requested.. cannot process:
> > > 67108864
> > > [5]PETSC ERROR: See https://urldefense.proofpoint.
> com/v2/url?u=http-3A__www.mcs.anl.gov_petsc_documentation_
> faq.html&d=DQIFAg&c=54IZrppPQZKX9mLzcGdPfFD1hxrcB_
> _aEkJFOKJFd00&r=DUUt3SRGI0_JgtNaS3udV68GRkgV4ts7XKfj2opmiCY&m=QrWHgIE8D6e-
> Soe4EmaMtPH316RnhxnYdz3z8-0BzAI&s=OT0cBiLgLS78J3C-
> bfKA5qWjJhVZ6ECrPajQbNri5BE&e=  for
> >
> > > trouble shooting.
> > > [5]PETSC ERROR: Petsc Development GIT revision: v3.7.4-2584-g7175242
> GIT
> > > Date: 2016-12-21 13:19:16 -0600
> > > [5]PETSC ERROR: /tmp/targetExec on a arch-xc30-dbg64-intel named
> nid00013
> > > by u6338 Sat Jan  7 18:35:34 2017
> > > [5]PETSC ERROR: Configure options --COPTFLAGS="-O0 -g -no-ipo"
> > > --CXXOPTFLAGS="-O0 -g -no-ipo" --FOPTFLAGS="-O0 -g -ipo"
> > > --download-parmetis --download-metis --download-hypre --with-ssl=0
> > > --with-cc=cc --with-clib-autodetect=0 --with-cxx=CC
> > > --with-cxxlib-autodetect=0 --with-debugging=1 --with-fc=ftn
> > > --with-fortranlib-autodetect=0 --with-shared-libraries=0 --with-x=0
> > > --with-mpiexec=srun LIBS=-lstdc++ --with-64-bit-indices
> > > --prefix=/global/homes/m/madams/petsc_install/petscv3.7-xc3[1983]PETSC
> > > ERROR: --------------------- Error Message
> ------------------------------
> > > --------------------------------
> > >
> > > [5]PETSC ERROR: [522]PETSC ERROR: --------------------- Error Message
> > > ----------
> > >
> > > [5]PETSC ERROR: #2 PetscTableCreate() line 52 in
> > > /global/u2/m/madams/petsc_install/petsc/src/sys/utils/ctable.c
> > > [5]PETSC ERROR: #3 MatTransposeMatMultSymbolic_MPIAIJ_MPIAIJ() line
> 1691 in
> > > /global/u2/m/madams/petsc_install/petsc/src/mat/impls/aij/
> > > mpi/mpimatmatmult.c
> > > [5]PETSC ERROR: #4 MatTransposeMatMult_MPIAIJ_MPIAIJ() line 917 in
> > > /global/u2/m/madams/petsc_install/petsc/src/mat/impls/aij/
> > > mpi/mpimatmatmult.c
> > > [5]PETSC ERROR: [608]PETSC ERROR: --------------------- Error Message
> > > ----------
> > >
> > > [5]PETSC ERROR: #6 PCGAMGCoarsen_AGG() line 955 in
> > > /global/u2/m/madams/petsc_install/petsc/src/ksp/pc/impls/gamg/agg.c
> > > [5]PETSC ERROR: #7 PCSetUp_GAMG() line 524 in
> /global/u2/m/madams/petsc_inst
> > > all/petsc/src/ksp/pc/impls/gamg/gamg.c
> > > [5]PETSC ERROR: #8 PCSetUp() line 968 in /global/u2/m/madams/petsc_inst
> > > all/petsc/src/ksp/pc/interface/precon.c
> > > [5]PETSC ERROR: #9 KSPSetUp() line 393 in
> /global/u2/m/madams/petsc_inst
> > > all/petsc/src/ksp/ksp/interface/itfunc.c
> > > [5]PETSC ERROR: [1640]PETSC ERROR: --------------------- Error Message
> -----
> > >
> > >
> > > > MatSetUpMultiply_MPIAIJ() does:
> > > >
> > > >   ierr = PetscTableCreate(aij->B->rmap-
> >n,mat->cmap->N+1,&gid1_lid1);
> > > > CHKERRQ(ierr);
> > > >
> > > > So the local matrix size (n) must be: 4294967296
> > > >
> > > > [assuming atleast one nonzero per row - the local matrix itself must
> use
> > > > more memory than the hash-table..]
> > > >
> > > > Satish
> > > >
> > >
> >
> >
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20170111/5fc94dbb/attachment.html>