[petsc-dev] Error on large problems.

[Mark Adams]

It works for a stripped down GAMG solve (no smoothing and no square
graph).  I will try adding the square graph back in ...

On Wed, Mar 25, 2015 at 7:24 AM, Mark Adams <mfadams at lbl.gov> wrote:

>
>
> On Sat, Mar 7, 2015 at 5:49 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
>
>>
>> > On Mar 7, 2015, at 4:27 PM, Mark Adams <mfadams at lbl.gov> wrote:
>> >
>> >
>> >
>> > On Sat, Mar 7, 2015 at 3:11 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
>> >
>> >   Hmm,  my first guess is a mixup between PetscMPIInt and PetscInt
>> arrays  or MPIU_INT somewhere. (But compilers catch most of these)
>> >
>> > I have a run with ~9K eq/core 128K cores many times but have never
>> gotten the 32K/core (4B eq.) run to work.  So it looks like an int overflow
>> issue.
>>
>>   Well in theory if we have done everything write with 64 bit indices
>> there should never be an integer overflow (though of course there could be
>> mistake somewhere but generally the compiler will detect if we are trying
>> to stick a 64 bit int into a 32 bit slot.
>>
>>    Can you try the same example without GAMG (say with hypre instead); if
>> it goes through ok it might indicate an int issue either in gamg or the
>> code that gamg calls?
>>
>>
> Good idea,  Jacobi works.  I will try a stripped down GAMG next.
>
>
>
>
>
>>   Barry
>>
>> >
>> >
>> >   Another possibility is bug in the MPI for many large messages; any
>> chance you can run the same thing on a very different system? Mira?
>> >
>> > Not soon, but I have Chombo and PETSc built on Mira and it would not be
>> hard to get this code setup and try it.
>> >
>> > This is for SCE15 so I will turn the PETSc test off unless someone has
>> any ideas on something to try.  I am using maint perhaps I should use
>> master.
>> >
>> > Mark
>> >
>> >
>> >   Barry
>> >
>> > > On Mar 7, 2015, at 1:21 PM, Mark Adams <mfadams at lbl.gov> wrote:
>> > >
>> > > I seem to be getting this error on Edison with 128K cores and ~4
>> Billion equations.  I've seen this error several time.  I've attached a
>> recent output from this.  I wonder if it is an integer overflow.  This
>> built with 64 bit integers, but I notice that GAMG prints out N and I see
>> N=0 for the finest level.
>> > >
>> > > Mark
>> > > <out.131072.uniform.txt>
>> >
>> >
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20150328/31feab54/attachment.html>