[petsc-dev] Fwd: no petsc on Edison

[Kong, Fande]

We do not run the code with Satish's fixes, instead, we are using
petsc-3.7.3 and there are not such issues. I guess the issues are
introduced recently.

Fande,

On Wed, Jan 11, 2017 at 1:31 PM, Mark Adams <mfadams at lbl.gov> wrote:

> Argh, yes that has to be the local size or a least not the global size.
>
> On Wed, Jan 11, 2017 at 3:15 PM, Satish Balay <balay at mcs.anl.gov> wrote:
>
>> >>>
>> aN=A->cmap->N
>> ierr = PetscTableCreate(aN,aN,&ta);CHKERRQ(ierr);
>> <<<
>>
>> Looks like thats a 'global' size. That doesn't look right.
>>
>> https://bitbucket.org/petsc/petsc/commits/c373ccc6#chg-src/
>> mat/impls/aij/mpi/mpimatmatmult.c
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__bitbucket.org_petsc_petsc_commits_c373ccc6-23chg-2Dsrc_mat_impls_aij_mpi_mpimatmatmult.c&d=DQMFaQ&c=54IZrppPQZKX9mLzcGdPfFD1hxrcB__aEkJFOKJFd00&r=DUUt3SRGI0_JgtNaS3udV68GRkgV4ts7XKfj2opmiCY&m=76yx34-qrsxdRv22O3eNUMz1k5Rzht2ZR7_1ayoSMEA&s=AG50mmcKH8CL_v5UwMmU0KnwLITrlarTj2_SJmVm3mU&e=>
>>
>> I see Hong had previously updated this code. Will have to recheck..
>>
>> Satish
>>
>> On Wed, 11 Jan 2017, Mark Adams wrote:
>>
>> > >
>> > >
>> > > Must be the same place Fande encountered this error.
>> > >
>> > >
>> > Satish, here is our stack trace. Do you have any thoughts vis-a-vis
>> Fande's
>> > experience what the problem is here?  My best guess is that the local
>> > matrix size is huge, but this code can have some very large stencils.
>> >
>> > Treb: about how many processors have this error? Is is it just one, a
>> few,
>> > or tons?
>> >
>> > [5]PETSC ERROR: Argument out of range
>> > [5]PETSC ERROR: A really huge hash is being requested.. cannot process:
>> > 67108864
>> > [5]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.mcs.anl.gov_petsc_documentation_faq.html&d=DQMFaQ&c=54IZrppPQZKX9mLzcGdPfFD1hxrcB__aEkJFOKJFd00&r=DUUt3SRGI0_JgtNaS3udV68GRkgV4ts7XKfj2opmiCY&m=76yx34-qrsxdRv22O3eNUMz1k5Rzht2ZR7_1ayoSMEA&s=YJkJ5MMR_SQlxpr5ivNKPKyG2u9uAvzj8imzwEuWVu8&e=>
>> for
>>
>> > trouble shooting.
>> > [5]PETSC ERROR: Petsc Development GIT revision: v3.7.4-2584-g7175242 GIT
>> > Date: 2016-12-21 13:19:16 -0600
>> > [5]PETSC ERROR: /tmp/targetExec on a arch-xc30-dbg64-intel named
>> nid00013
>> > by u6338 Sat Jan  7 18:35:34 2017
>> > [5]PETSC ERROR: Configure options --COPTFLAGS="-O0 -g -no-ipo"
>> > --CXXOPTFLAGS="-O0 -g -no-ipo" --FOPTFLAGS="-O0 -g -ipo"
>> > --download-parmetis --download-metis --download-hypre --with-ssl=0
>> > --with-cc=cc --with-clib-autodetect=0 --with-cxx=CC
>> > --with-cxxlib-autodetect=0 --with-debugging=1 --with-fc=ftn
>> > --with-fortranlib-autodetect=0 --with-shared-libraries=0 --with-x=0
>> > --with-mpiexec=srun LIBS=-lstdc++ --with-64-bit-indices
>> > --prefix=/global/homes/m/madams/petsc_install/petscv3.7-xc3[1983]PETSC
>> > ERROR: --------------------- Error Message
>> ------------------------------
>> > --------------------------------
>> >
>> > [5]PETSC ERROR: [522]PETSC ERROR: --------------------- Error Message
>> > ----------
>> >
>> > [5]PETSC ERROR: #2 PetscTableCreate() line 52 in
>> > /global/u2/m/madams/petsc_install/petsc/src/sys/utils/ctable.c
>> > [5]PETSC ERROR: #3 MatTransposeMatMultSymbolic_MPIAIJ_MPIAIJ() line
>> 1691 in
>> > /global/u2/m/madams/petsc_install/petsc/src/mat/impls/aij/
>> > mpi/mpimatmatmult.c
>> > [5]PETSC ERROR: #4 MatTransposeMatMult_MPIAIJ_MPIAIJ() line 917 in
>> > /global/u2/m/madams/petsc_install/petsc/src/mat/impls/aij/
>> > mpi/mpimatmatmult.c
>> > [5]PETSC ERROR: [608]PETSC ERROR: --------------------- Error Message
>> > ----------
>> >
>> > [5]PETSC ERROR: #6 PCGAMGCoarsen_AGG() line 955 in
>> > /global/u2/m/madams/petsc_install/petsc/src/ksp/pc/impls/gamg/agg.c
>> > [5]PETSC ERROR: #7 PCSetUp_GAMG() line 524 in
>> /global/u2/m/madams/petsc_inst
>> > all/petsc/src/ksp/pc/impls/gamg/gamg.c
>> > [5]PETSC ERROR: #8 PCSetUp() line 968 in /global/u2/m/madams/petsc_inst
>> > all/petsc/src/ksp/pc/interface/precon.c
>> > [5]PETSC ERROR: #9 KSPSetUp() line 393 in /global/u2/m/madams/petsc_inst
>> > all/petsc/src/ksp/ksp/interface/itfunc.c
>> > [5]PETSC ERROR: [1640]PETSC ERROR: --------------------- Error Message
>> -----
>> >
>> >
>> > > MatSetUpMultiply_MPIAIJ() does:
>> > >
>> > >   ierr = PetscTableCreate(aij->B->rmap->n,mat->cmap->N+1,&gid1_lid1);
>> > > CHKERRQ(ierr);
>> > >
>> > > So the local matrix size (n) must be: 4294967296
>> > >
>> > > [assuming atleast one nonzero per row - the local matrix itself must
>> use
>> > > more memory than the hash-table..]
>> > >
>> > > Satish
>> > >
>> >
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20170111/1f32c173/attachment.html>