[petsc-users] KSPSolve errors on blues

[Xujun Zhao]

A quick test of a smaller system on my laptop with 25x25x25 mesh gives the
following info.
The memory used keeps increasing from 1 to 3 CPUs, but slightly decreases
with 4 CPUs.
On the other hand, 60x60x60 mesh (2.9M dofs) is also not a big system...


------------------------------------------------- 1 CPU
-------------------------------------------------
Summary of Memory Usage in PETSc
Maximum (over computational time) process memory:        total 4.7054e+09
max 4.7054e+09 min 4.7054e+09
Current process memory:
total 4.7054e+09 max 4.7054e+09 min 4.7054e+09
Maximum (over computational time) space PetscMalloc()ed: total 1.6151e+09
max 1.6151e+09 min 1.6151e+09
Current space PetscMalloc()ed:
total 7.7232e+04 max 7.7232e+04 min 7.7232e+04

------------------------------------------------- 2 CPU
-------------------------------------------------
Summary of Memory Usage in PETSc
Maximum (over computational time) process memory:        total 6.2389e+09
max 3.1275e+09 min 3.1113e+09
Current process memory:
total 6.2389e+09 max 3.1275e+09 min 3.1113e+09
Maximum (over computational time) space PetscMalloc()ed: total 2.1589e+09
max 1.1193e+09 min 1.0397e+09
Current space PetscMalloc()ed:
total 1.5446e+05 max 7.7232e+04 min 7.7232e+04

------------------------------------------------- 3 CPU
-------------------------------------------------
Summary of Memory Usage in PETSc
Maximum (over computational time) process memory:        total 7.7116e+09
max 1.9572e+09 min 1.8715e+09
Current process memory:
total 7.7116e+09 max 1.9572e+09 min 1.8715e+09
Maximum (over computational time) space PetscMalloc()ed: total 2.1754e+09
max 5.8450e+08 min 5.0516e+08
Current space PetscMalloc()ed:
total 3.0893e+05 max 7.7232e+04 min 7.7232e+04

------------------------------------------------- 4 CPU
-------------------------------------------------
Summary of Memory Usage in PETSc
Maximum (over computational time) process memory:        total 7.1188e+09
max 2.4651e+09 min 2.2909e+09
Current process memory:
total 7.1188e+09 max 2.4651e+09 min 2.2909e+09
Maximum (over computational time) space PetscMalloc()ed: total 2.1750e+09
max 7.6982e+08 min 6.5289e+08
Current space PetscMalloc()ed:
total 2.3170e+05 max 7.7232e+04 min 7.7232e+04


On Wed, Jun 8, 2016 at 11:51 AM, Barry Smith <bsmith at mcs.anl.gov> wrote:

>
>    Signal 9 SIGKILL on batch systems usually means the process was killed
> because it ran out of time or ran out of memory.
>
>     Perhaps there is something in the code that is unscalable and requires
> more more memory with more processes. You can run on 1 2 and 3 processes
> and measure the memory usage to see if it goes up with the number of
> processes using for example -memory_view
>
>   Barry
>
>
>
> > On Jun 8, 2016, at 11:41 AM, Xujun Zhao <xzhao99 at gmail.com> wrote:
> >
> > Hi all,
> >
> > I am running a FE Stokes Solver with schur complement type PC on blues.
> The program runs well when mesh is 40X40X40 (0.88M dofs), but when I use
> 60X60X60 mesh, the program crashes and gives out some errors, which looks
> like a segmentation fault. The "strange" thing is that it runs well with
> 1CPU, 2CPUs, but fails on 4 or 8 CPUs. The log files are also attached. It
> seems like the global matrix and vector are assembled well, and errors come
> out before calling the KSPSolve().
> >
> > btw, I use the recent PETSc 3.7 dbg version. for libMesh I use both dbg
> and opt version, but none of those can give useful information. Has anyone
> met such situations before? Many thinks.
> > <ex01_validation_test.o1386940><ex01_validation_test.o1387007>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20160608/a17c18e8/attachment.html>