[petsc-dev] making DA more light weight
Jed Brown
jed at jedbrown.org
Thu May 15 17:38:28 CDT 2014
Barry Smith <bsmith at mcs.anl.gov> writes:
>> Note that ISGetIndices is still called in the parallel case.
>
> When bs is set > 1 it is not called in the VecScatterCreate()
> inside the DMDACreate 2d/3d ! Only ISBlockGetIndices() is
> called. The memory usage of VecScatterCreate() is O(dof*number of
> ghost points) + O(vector length/dof) in this case
>
> It is called MULTIPLY times in the ISLocalToGlobalMappingCreateIS()
> inside the DMDACreate 2d/3d thus currently this causes small integer
> * vector length usage. As I keep telling you this is the problem
> area for dof > 1. For dof == 1 I can live with 2.5 *sizeof (vector)
>
> You need a better test code to run with. Can you use
> ksp/ksp/examples/tests/ex42.c ? Or snes/examples/tutorials/ex19.c
> in parallel?
I added a simple test for this purpose, src/dm/examples/tests/ex20.c
in 'jed/dmda-memusage'.
$ mpirun.hydra -n 2 ./ex20 -malloc_log -malloc_log_threshold 1e6
DMDACreate3d : 225.866 MB + 225.866 MB
DMCreateGlobalVector: 309.666 MB + 83.800 MB
DMCreateGlobalVector: 393.552 MB + 83.886 MB
[0] Maximum memory PetscMalloc()ed 386401520 maximum size of entire process 309821440
[0] Memory usage sorted by function
[0] 3 12713984 DMSetUp_DA_3D()
[0] 1 4259840 ISBlockSetIndices_Block()
[0] 4 169738240 ISGetIndices_Block()
[0] 1 41943040 ISGetIndices_Stride()
[0] 1 4259840 ISLocalToGlobalMappingBlock()
[0] 2 167772160 VecCreate_MPI_Private()
[0] 10 339476480 VecScatterCreate_PtoS()
If you run as above in a debugger with a break point at ISGetIndices,
the first break is at
$ mpirun.hydra -n 1 gdb -ex 'b ISGetIndices' -ex r --args ./ex20 : -n 1 ./ex20
GNU gdb (GDB) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
No symbol table is loaded. Use the "file" command.
Breakpoint 1 (err.c:343) pending.
Reading symbols from ./ex20...done.
Function "ISGetIndices" not defined.
Breakpoint 2 (ISGetIndices) pending.
Starting program: /home/jed/petsc/src/dm/examples/tests/ex20
warning: Could not load shared library symbols for linux-vdso.so.1.
Do you need "set solib-search-path" or "set sysroot"?
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
Breakpoint 2, ISGetIndices (is=0x506840, ptr=0x7fffffffd328) at /home/jed/petsc/src/vec/is/is/interface/index.c:379
379 PetscFunctionBegin;
(gdb) bt
#0 ISGetIndices (is=0x506840, ptr=0x7fffffffd328) at /home/jed/petsc/src/vec/is/is/interface/index.c:379
#1 0x00007ffff6a60cb9 in VecScatterCreate (xin=0x4fbfb0, ix=0x506840, yin=0x4f8d10, iy=0x503210, newctx=0x7fffffffd888) at /home/jed/petsc/src/vec/vec/utils/vscat.c:1579
#2 0x00007ffff7129d9b in DMSetUp_DA_3D (da=0x4e3280) at /home/jed/petsc/src/dm/impls/da/da3.c:448
#3 0x00007ffff71a3ef5 in DMSetUp_DA (da=0x4e3280) at /home/jed/petsc/src/dm/impls/da/dareg.c:27
#4 0x00007ffff736fd9f in DMSetUp (dm=0x4e3280) at /home/jed/petsc/src/dm/interface/dm.c:559
#5 0x00007ffff71330b1 in DMDACreate3d (comm=1140850688, bx=DM_BOUNDARY_NONE, by=DM_BOUNDARY_NONE, bz=DM_BOUNDARY_NONE, stencil_type=DMDA_STENCIL_STAR, M=-128, N=-128, P=-128, m=-1, n=-1, p=-1, dof=10, s=1, lx=0x0, ly=0x0, lz=0x0, da=0x7fffffffdd58) at /home/jed/petsc/src/dm/impls/da/da3.c:1463
#6 0x0000000000400c96 in main (argc=1, argv=0x7fffffffde58) at /home/jed/petsc/src/dm/examples/tests/ex20.c:15
That is this code, which dates from 1995 according to git annotate.
/* general case */
{
PetscInt nx,ny;
const PetscInt *idx,*idy;
ierr = ISGetLocalSize(ix,&nx);CHKERRQ(ierr);
ierr = ISGetIndices(ix,&idx);CHKERRQ(ierr);
ierr = ISGetLocalSize(iy,&ny);CHKERRQ(ierr);
ierr = ISGetIndices(iy,&idy);CHKERRQ(ierr);
if (nx != ny) SETERRQ(PETSC_COMM_SELF,PETSC_ERR_ARG_SIZ,"Local scatter sizes don't match");
ierr = VecScatterCreate_StoP(nx,idx,ny,idy,xin,yin,1,ctx);CHKERRQ(ierr);
ierr = ISRestoreIndices(ix,&idx);CHKERRQ(ierr);
ierr = ISRestoreIndices(iy,&idy);CHKERRQ(ierr);
ierr = PetscInfo(xin,"General case: Seq to MPI\n");CHKERRQ(ierr);
goto functionend;
}
Here are all the large allocations, found by stepping through printing
the stack trace for each case.
$ mpirun.hydra -n 1 gdb -ex 'b mtr.c:228' -ex r --args ./ex20 -malloc_log -malloc_log_threshold 1e7 : -n 1 ./ex20
[...]
(gdb) bt
#0 PetscTrMallocDefault (a=41943040, lineno=47, function=0x7ffff7afd5d4 <.L__func__.ISGetIndices_Block> "ISGetIndices_Block", filename=0x7ffff7afd53b <.L.str> "/home/jed/petsc/src/vec/is/is/impls/block/block.c", result=0x7fffffffcf88) at /home/jed/petsc/src/sys/memory/mtr.c:228
#1 0x00007ffff691fad2 in ISGetIndices_Block (in=0x4cbe90, idx=0x7fffffffd2f8) at /home/jed/petsc/src/vec/is/is/impls/block/block.c:47
#2 0x00007ffff6936468 in ISGetIndices (is=0x4cbe90, ptr=0x7fffffffd2f8) at /home/jed/petsc/src/vec/is/is/interface/index.c:382
#3 0x00007ffff6a60cb9 in VecScatterCreate (xin=0x4c1600, ix=0x4cbe90, yin=0x4be360, iy=0x4c8860, newctx=0x7fffffffd858) at /home/jed/petsc/src/vec/vec/utils/vscat.c:1579
#4 0x00007ffff7129d9b in DMSetUp_DA_3D (da=0x4a88d0) at /home/jed/petsc/src/dm/impls/da/da3.c:448
#5 0x00007ffff71a3ef5 in DMSetUp_DA (da=0x4a88d0) at /home/jed/petsc/src/dm/impls/da/dareg.c:27
#6 0x00007ffff736fd9f in DMSetUp (dm=0x4a88d0) at /home/jed/petsc/src/dm/interface/dm.c:559
#7 0x00007ffff71330b1 in DMDACreate3d (comm=1140850688, bx=DM_BOUNDARY_NONE, by=DM_BOUNDARY_NONE, bz=DM_BOUNDARY_NONE, stencil_type=DMDA_STENCIL_STAR, M=-128, N=-128, P=-128, m=-1, n=-1, p=-1, dof=10, s=1, lx=0x0, ly=0x0, lz=0x0, da=0x7fffffffdd28) at /home/jed/petsc/src/dm/impls/da/da3.c:1463
#8 0x0000000000400c96 in main (argc=4, argv=0x7fffffffde28) at /home/jed/petsc/src/dm/examples/tests/ex20.c:15
#0 PetscTrMallocDefault (a=41943040, lineno=160, function=0x7ffff7afd32b <.L__func__.ISGetIndices_Stride> "ISGetIndices_Stride", filename=0x7ffff7afd279 <.L.str> "/home/jed/petsc/src/vec/is/is/impls/stride/stride.c", result=0x7fffffffd2f0) at /home/jed/petsc/src/sys/memory/mtr.c:228
#1 0x00007ffff691b1e9 in ISGetIndices_Stride (in=0x4c8860, idx=0x7fffffffd2f0) at /home/jed/petsc/src/vec/is/is/impls/stride/stride.c:160
#2 0x00007ffff6936468 in ISGetIndices (is=0x4c8860, ptr=0x7fffffffd2f0) at /home/jed/petsc/src/vec/is/is/interface/index.c:382
#3 0x00007ffff6a60da7 in VecScatterCreate (xin=0x4c1600, ix=0x4cbe90, yin=0x4be360, iy=0x4c8860, newctx=0x7fffffffd858) at /home/jed/petsc/src/vec/vec/utils/vscat.c:1581
#4 0x00007ffff7129d9b in DMSetUp_DA_3D (da=0x4a88d0) at /home/jed/petsc/src/dm/impls/da/da3.c:448
#5 0x00007ffff71a3ef5 in DMSetUp_DA (da=0x4a88d0) at /home/jed/petsc/src/dm/impls/da/dareg.c:27
#6 0x00007ffff736fd9f in DMSetUp (dm=0x4a88d0) at /home/jed/petsc/src/dm/interface/dm.c:559
#7 0x00007ffff71330b1 in DMDACreate3d (comm=1140850688, bx=DM_BOUNDARY_NONE, by=DM_BOUNDARY_NONE, bz=DM_BOUNDARY_NONE, stencil_type=DMDA_STENCIL_STAR, M=-128, N=-128, P=-128, m=-1, n=-1, p=-1, dof=10, s=1, lx=0x0, ly=0x0, lz=0x0, da=0x7fffffffdd28) at /home/jed/petsc/src/dm/impls/da/da3.c:1463
#8 0x0000000000400c96 in main (argc=4, argv=0x7fffffffde28) at /home/jed/petsc/src/dm/examples/tests/ex20.c:15
#0 PetscTrMallocDefault (a=41943040, lineno=1745, function=0x7ffff7b0449a <.L__func__.VecScatterCreate_PtoS> "VecScatterCreate_PtoS", filename=0x7ffff7b03f29 <.L.str> "/home/jed/petsc/src/vec/vec/utils/vpscat.c", result=0x7fffffffcee0) at /home/jed/petsc/src/sys/memory/mtr.c:228
#1 0x00007ffff6a9129f in VecScatterCreate_PtoS (nx=10485760, inidx=0x7fffe9086650, ny=10485760, inidy=0x7fffeb887650, xin=0x4be360, yin=0x4c1600, bs=1, ctx=0x4d2ea0) at /home/jed/petsc/src/vec/vec/utils/vpscat.c:1745
#2 0x00007ffff6a94c2d in VecScatterCreate_StoP (nx=10485760, inidx=0x7fffeb887650, ny=10485760, inidy=0x7fffe9086650, xin=0x4c1600, yin=0x4be360, bs=1, ctx=0x4d2ea0) at /home/jed/petsc/src/vec/vec/utils/vpscat.c:2197
#3 0x00007ffff6a60ea0 in VecScatterCreate (xin=0x4c1600, ix=0x4cbe90, yin=0x4be360, iy=0x4c8860, newctx=0x7fffffffd858) at /home/jed/petsc/src/vec/vec/utils/vscat.c:1583
#4 0x00007ffff7129d9b in DMSetUp_DA_3D (da=0x4a88d0) at /home/jed/petsc/src/dm/impls/da/da3.c:448
#5 0x00007ffff71a3ef5 in DMSetUp_DA (da=0x4a88d0) at /home/jed/petsc/src/dm/impls/da/dareg.c:27
#6 0x00007ffff736fd9f in DMSetUp (dm=0x4a88d0) at /home/jed/petsc/src/dm/interface/dm.c:559
#7 0x00007ffff71330b1 in DMDACreate3d (comm=1140850688, bx=DM_BOUNDARY_NONE, by=DM_BOUNDARY_NONE, bz=DM_BOUNDARY_NONE, stencil_type=DMDA_STENCIL_STAR, M=-128, N=-128, P=-128, m=-1, n=-1, p=-1, dof=10, s=1, lx=0x0, ly=0x0, lz=0x0, da=0x7fffffffdd28) at /home/jed/petsc/src/dm/impls/da/da3.c:1463
#8 0x0000000000400c96 in main (argc=4, argv=0x7fffffffde28) at /home/jed/petsc/src/dm/examples/tests/ex20.c:15
#0 PetscTrMallocDefault (a=41943040, lineno=1784, function=0x7ffff7b0449a <.L__func__.VecScatterCreate_PtoS> "VecScatterCreate_PtoS", filename=0x7ffff7b03f29 <.L.str> "/home/jed/petsc/src/vec/vec/utils/vpscat.c", result=0x7fffffffcec0) at /home/jed/petsc/src/sys/memory/mtr.c:228
#1 0x00007ffff6a91c5f in VecScatterCreate_PtoS (nx=10485760, inidx=0x7fffe9086650, ny=10485760, inidy=0x7fffeb887650, xin=0x4be360, yin=0x4c1600, bs=1, ctx=0x4d2ea0) at /home/jed/petsc/src/vec/vec/utils/vpscat.c:1784
#2 0x00007ffff6a94c2d in VecScatterCreate_StoP (nx=10485760, inidx=0x7fffeb887650, ny=10485760, inidy=0x7fffe9086650, xin=0x4c1600, yin=0x4be360, bs=1, ctx=0x4d2ea0) at /home/jed/petsc/src/vec/vec/utils/vpscat.c:2197
#3 0x00007ffff6a60ea0 in VecScatterCreate (xin=0x4c1600, ix=0x4cbe90, yin=0x4be360, iy=0x4c8860, newctx=0x7fffffffd858) at /home/jed/petsc/src/vec/vec/utils/vscat.c:1583
#4 0x00007ffff7129d9b in DMSetUp_DA_3D (da=0x4a88d0) at /home/jed/petsc/src/dm/impls/da/da3.c:448
#5 0x00007ffff71a3ef5 in DMSetUp_DA (da=0x4a88d0) at /home/jed/petsc/src/dm/impls/da/dareg.c:27
#6 0x00007ffff736fd9f in DMSetUp (dm=0x4a88d0) at /home/jed/petsc/src/dm/interface/dm.c:559
#7 0x00007ffff71330b1 in DMDACreate3d (comm=1140850688, bx=DM_BOUNDARY_NONE, by=DM_BOUNDARY_NONE, bz=DM_BOUNDARY_NONE, stencil_type=DMDA_STENCIL_STAR, M=-128, N=-128, P=-128, m=-1, n=-1, p=-1, dof=10, s=1, lx=0x0, ly=0x0, lz=0x0, da=0x7fffffffdd28) at /home/jed/petsc/src/dm/impls/da/da3.c:1463
#8 0x0000000000400c96 in main (argc=4, argv=0x7fffffffde28) at /home/jed/petsc/src/dm/examples/tests/ex20.c:15
#0 PetscTrMallocDefault (a=41943040, lineno=1879, function=0x7ffff7b0449a <.L__func__.VecScatterCreate_PtoS> "VecScatterCreate_PtoS", filename=0x7ffff7b03f29 <.L.str> "/home/jed/petsc/src/vec/vec/utils/vpscat.c", result=0x50fae0) at /home/jed/petsc/src/sys/memory/mtr.c:228
#1 0x00007ffff6a9423b in VecScatterCreate_PtoS (nx=10485760, inidx=0x7fffe9086650, ny=10485760, inidy=0x7fffeb887650, xin=0x4be360, yin=0x4c1600, bs=1, ctx=0x4d2ea0) at /home/jed/petsc/src/vec/vec/utils/vpscat.c:1879
#2 0x00007ffff6a94c2d in VecScatterCreate_StoP (nx=10485760, inidx=0x7fffeb887650, ny=10485760, inidy=0x7fffe9086650, xin=0x4c1600, yin=0x4be360, bs=1, ctx=0x4d2ea0) at /home/jed/petsc/src/vec/vec/utils/vpscat.c:2197
#3 0x00007ffff6a60ea0 in VecScatterCreate (xin=0x4c1600, ix=0x4cbe90, yin=0x4be360, iy=0x4c8860, newctx=0x7fffffffd858) at /home/jed/petsc/src/vec/vec/utils/vscat.c:1583
#4 0x00007ffff7129d9b in DMSetUp_DA_3D (da=0x4a88d0) at /home/jed/petsc/src/dm/impls/da/da3.c:448
#5 0x00007ffff71a3ef5 in DMSetUp_DA (da=0x4a88d0) at /home/jed/petsc/src/dm/impls/da/dareg.c:27
#6 0x00007ffff736fd9f in DMSetUp (dm=0x4a88d0) at /home/jed/petsc/src/dm/interface/dm.c:559
#7 0x00007ffff71330b1 in DMDACreate3d (comm=1140850688, bx=DM_BOUNDARY_NONE, by=DM_BOUNDARY_NONE, bz=DM_BOUNDARY_NONE, stencil_type=DMDA_STENCIL_STAR, M=-128, N=-128, P=-128, m=-1, n=-1, p=-1, dof=10, s=1, lx=0x0, ly=0x0, lz=0x0, da=0x7fffffffdd28) at /home/jed/petsc/src/dm/impls/da/da3.c:1463
#8 0x0000000000400c96 in main (argc=4, argv=0x7fffffffde28) at /home/jed/petsc/src/dm/examples/tests/ex20.c:15
#0 PetscTrMallocDefault (a=41943040, lineno=1880, function=0x7ffff7b0449a <.L__func__.VecScatterCreate_PtoS> "VecScatterCreate_PtoS", filename=0x7ffff7b03f29 <.L.str> "/home/jed/petsc/src/vec/vec/utils/vpscat.c", result=0x510860) at /home/jed/petsc/src/sys/memory/mtr.c:228
#1 0x00007ffff6a9431f in VecScatterCreate_PtoS (nx=10485760, inidx=0x7fffe9086650, ny=10485760, inidy=0x7fffeb887650, xin=0x4be360, yin=0x4c1600, bs=1, ctx=0x4d2ea0) at /home/jed/petsc/src/vec/vec/utils/vpscat.c:1880
#2 0x00007ffff6a94c2d in VecScatterCreate_StoP (nx=10485760, inidx=0x7fffeb887650, ny=10485760, inidy=0x7fffe9086650, xin=0x4c1600, yin=0x4be360, bs=1, ctx=0x4d2ea0) at /home/jed/petsc/src/vec/vec/utils/vpscat.c:2197
#3 0x00007ffff6a60ea0 in VecScatterCreate (xin=0x4c1600, ix=0x4cbe90, yin=0x4be360, iy=0x4c8860, newctx=0x7fffffffd858) at /home/jed/petsc/src/vec/vec/utils/vscat.c:1583
#4 0x00007ffff7129d9b in DMSetUp_DA_3D (da=0x4a88d0) at /home/jed/petsc/src/dm/impls/da/da3.c:448
#5 0x00007ffff71a3ef5 in DMSetUp_DA (da=0x4a88d0) at /home/jed/petsc/src/dm/impls/da/dareg.c:27
#6 0x00007ffff736fd9f in DMSetUp (dm=0x4a88d0) at /home/jed/petsc/src/dm/interface/dm.c:559
#7 0x00007ffff71330b1 in DMDACreate3d (comm=1140850688, bx=DM_BOUNDARY_NONE, by=DM_BOUNDARY_NONE, bz=DM_BOUNDARY_NONE, stencil_type=DMDA_STENCIL_STAR, M=-128, N=-128, P=-128, m=-1, n=-1, p=-1, dof=10, s=1, lx=0x0, ly=0x0, lz=0x0, da=0x7fffffffdd28) at /home/jed/petsc/src/dm/impls/da/da3.c:1463
#8 0x0000000000400c96 in main (argc=4, argv=0x7fffffffde28) at /home/jed/petsc/src/dm/examples/tests/ex20.c:15
#0 PetscTrMallocDefault (a=42598400, lineno=47, function=0x7ffff7afd5d4 <.L__func__.ISGetIndices_Block> "ISGetIndices_Block", filename=0x7ffff7afd53b <.L.str> "/home/jed/petsc/src/vec/is/is/impls/block/block.c", result=0x7fffffffcf88) at /home/jed/petsc/src/sys/memory/mtr.c:228
#1 0x00007ffff691fad2 in ISGetIndices_Block (in=0x4cf6a0, idx=0x7fffffffd370) at /home/jed/petsc/src/vec/is/is/impls/block/block.c:47
#2 0x00007ffff6936468 in ISGetIndices (is=0x4cf6a0, ptr=0x7fffffffd370) at /home/jed/petsc/src/vec/is/is/interface/index.c:382
#3 0x00007ffff6a5f84a in VecScatterCreate (xin=0x4be360, ix=0x4cf6a0, yin=0x4c1600, iy=0x50ede0, newctx=0x7fffffffd850) at /home/jed/petsc/src/vec/vec/utils/vscat.c:1493
#4 0x00007ffff712ed23 in DMSetUp_DA_3D (da=0x4a88d0) at /home/jed/petsc/src/dm/impls/da/da3.c:1033
#5 0x00007ffff71a3ef5 in DMSetUp_DA (da=0x4a88d0) at /home/jed/petsc/src/dm/impls/da/dareg.c:27
#6 0x00007ffff736fd9f in DMSetUp (dm=0x4a88d0) at /home/jed/petsc/src/dm/interface/dm.c:559
#7 0x00007ffff71330b1 in DMDACreate3d (comm=1140850688, bx=DM_BOUNDARY_NONE, by=DM_BOUNDARY_NONE, bz=DM_BOUNDARY_NONE, stencil_type=DMDA_STENCIL_STAR, M=-128, N=-128, P=-128, m=-1, n=-1, p=-1, dof=10, s=1, lx=0x0, ly=0x0, lz=0x0, da=0x7fffffffdd28) at /home/jed/petsc/src/dm/impls/da/da3.c:1463
#8 0x0000000000400c96 in main (argc=4, argv=0x7fffffffde28) at /home/jed/petsc/src/dm/examples/tests/ex20.c:15
#0 PetscTrMallocDefault (a=42598400, lineno=47, function=0x7ffff7afd5d4 <.L__func__.ISGetIndices_Block> "ISGetIndices_Block", filename=0x7ffff7afd53b <.L.str> "/home/jed/petsc/src/vec/is/is/impls/block/block.c", result=0x7fffffffcf88) at /home/jed/petsc/src/sys/memory/mtr.c:228
#1 0x00007ffff691fad2 in ISGetIndices_Block (in=0x50ede0, idx=0x7fffffffd368) at /home/jed/petsc/src/vec/is/is/impls/block/block.c:47
#2 0x00007ffff6936468 in ISGetIndices (is=0x50ede0, ptr=0x7fffffffd368) at /home/jed/petsc/src/vec/is/is/interface/index.c:382
#3 0x00007ffff6a5f938 in VecScatterCreate (xin=0x4be360, ix=0x4cf6a0, yin=0x4c1600, iy=0x50ede0, newctx=0x7fffffffd850) at /home/jed/petsc/src/vec/vec/utils/vscat.c:1495
#4 0x00007ffff712ed23 in DMSetUp_DA_3D (da=0x4a88d0) at /home/jed/petsc/src/dm/impls/da/da3.c:1033
#5 0x00007ffff71a3ef5 in DMSetUp_DA (da=0x4a88d0) at /home/jed/petsc/src/dm/impls/da/dareg.c:27
#6 0x00007ffff736fd9f in DMSetUp (dm=0x4a88d0) at /home/jed/petsc/src/dm/interface/dm.c:559
#7 0x00007ffff71330b1 in DMDACreate3d (comm=1140850688, bx=DM_BOUNDARY_NONE, by=DM_BOUNDARY_NONE, bz=DM_BOUNDARY_NONE, stencil_type=DMDA_STENCIL_STAR, M=-128, N=-128, P=-128, m=-1, n=-1, p=-1, dof=10, s=1, lx=0x0, ly=0x0, lz=0x0, da=0x7fffffffdd28) at /home/jed/petsc/src/dm/impls/da/da3.c:1463
#8 0x0000000000400c96 in main (argc=4, argv=0x7fffffffde28) at /home/jed/petsc/src/dm/examples/tests/ex20.c:15
#0 PetscTrMallocDefault (a=42598400, lineno=1745, function=0x7ffff7b0449a <.L__func__.VecScatterCreate_PtoS> "VecScatterCreate_PtoS", filename=0x7ffff7b03f29 <.L.str> "/home/jed/petsc/src/vec/vec/utils/vpscat.c", result=0x7fffffffcf80) at /home/jed/petsc/src/sys/memory/mtr.c:228
#1 0x00007ffff6a9129f in VecScatterCreate_PtoS (nx=10649600, inidx=0x7fffeafb5650, ny=10649600, inidy=0x7fffe17e3650, xin=0x4be360, yin=0x4c1600, bs=1, ctx=0x514f70) at /home/jed/petsc/src/vec/vec/utils/vpscat.c:1745
#2 0x00007ffff6a5fa49 in VecScatterCreate (xin=0x4be360, ix=0x4cf6a0, yin=0x4c1600, iy=0x50ede0, newctx=0x7fffffffd850) at /home/jed/petsc/src/vec/vec/utils/vscat.c:1497
#3 0x00007ffff712ed23 in DMSetUp_DA_3D (da=0x4a88d0) at /home/jed/petsc/src/dm/impls/da/da3.c:1033
#4 0x00007ffff71a3ef5 in DMSetUp_DA (da=0x4a88d0) at /home/jed/petsc/src/dm/impls/da/dareg.c:27
#5 0x00007ffff736fd9f in DMSetUp (dm=0x4a88d0) at /home/jed/petsc/src/dm/interface/dm.c:559
#6 0x00007ffff71330b1 in DMDACreate3d (comm=1140850688, bx=DM_BOUNDARY_NONE, by=DM_BOUNDARY_NONE, bz=DM_BOUNDARY_NONE, stencil_type=DMDA_STENCIL_STAR, M=-128, N=-128, P=-128, m=-1, n=-1, p=-1, dof=10, s=1, lx=0x0, ly=0x0, lz=0x0, da=0x7fffffffdd28) at /home/jed/petsc/src/dm/impls/da/da3.c:1463
#7 0x0000000000400c96 in main (argc=4, argv=0x7fffffffde28) at /home/jed/petsc/src/dm/examples/tests/ex20.c:15
#0 PetscTrMallocDefault (a=42598400, lineno=1784, function=0x7ffff7b0449a <.L__func__.VecScatterCreate_PtoS> "VecScatterCreate_PtoS", filename=0x7ffff7b03f29 <.L.str> "/home/jed/petsc/src/vec/vec/utils/vpscat.c", result=0x7fffffffcf60) at /home/jed/petsc/src/sys/memory/mtr.c:228
#1 0x00007ffff6a91c5f in VecScatterCreate_PtoS (nx=10649600, inidx=0x7fffeafb5650, ny=10649600, inidy=0x7fffe17e3650, xin=0x4be360, yin=0x4c1600, bs=1, ctx=0x514f70) at /home/jed/petsc/src/vec/vec/utils/vpscat.c:1784
#2 0x00007ffff6a5fa49 in VecScatterCreate (xin=0x4be360, ix=0x4cf6a0, yin=0x4c1600, iy=0x50ede0, newctx=0x7fffffffd850) at /home/jed/petsc/src/vec/vec/utils/vscat.c:1497
#3 0x00007ffff712ed23 in DMSetUp_DA_3D (da=0x4a88d0) at /home/jed/petsc/src/dm/impls/da/da3.c:1033
#4 0x00007ffff71a3ef5 in DMSetUp_DA (da=0x4a88d0) at /home/jed/petsc/src/dm/impls/da/dareg.c:27
#5 0x00007ffff736fd9f in DMSetUp (dm=0x4a88d0) at /home/jed/petsc/src/dm/interface/dm.c:559
#6 0x00007ffff71330b1 in DMDACreate3d (comm=1140850688, bx=DM_BOUNDARY_NONE, by=DM_BOUNDARY_NONE, bz=DM_BOUNDARY_NONE, stencil_type=DMDA_STENCIL_STAR, M=-128, N=-128, P=-128, m=-1, n=-1, p=-1, dof=10, s=1, lx=0x0, ly=0x0, lz=0x0, da=0x7fffffffdd28) at /home/jed/petsc/src/dm/impls/da/da3.c:1463
#7 0x0000000000400c96 in main (argc=4, argv=0x7fffffffde28) at /home/jed/petsc/src/dm/examples/tests/ex20.c:15
#0 PetscTrMallocDefault (a=41943040, lineno=1879, function=0x7ffff7b0449a <.L__func__.VecScatterCreate_PtoS> "VecScatterCreate_PtoS", filename=0x7ffff7b03f29 <.L.str> "/home/jed/petsc/src/vec/vec/utils/vpscat.c", result=0x5b9850) at /home/jed/petsc/src/sys/memory/mtr.c:228
#1 0x00007ffff6a9423b in VecScatterCreate_PtoS (nx=10649600, inidx=0x7fffeafb5650, ny=10649600, inidy=0x7fffe17e3650, xin=0x4be360, yin=0x4c1600, bs=1, ctx=0x514f70) at /home/jed/petsc/src/vec/vec/utils/vpscat.c:1879
#2 0x00007ffff6a5fa49 in VecScatterCreate (xin=0x4be360, ix=0x4cf6a0, yin=0x4c1600, iy=0x50ede0, newctx=0x7fffffffd850) at /home/jed/petsc/src/vec/vec/utils/vscat.c:1497
#3 0x00007ffff712ed23 in DMSetUp_DA_3D (da=0x4a88d0) at /home/jed/petsc/src/dm/impls/da/da3.c:1033
#4 0x00007ffff71a3ef5 in DMSetUp_DA (da=0x4a88d0) at /home/jed/petsc/src/dm/impls/da/dareg.c:27
#5 0x00007ffff736fd9f in DMSetUp (dm=0x4a88d0) at /home/jed/petsc/src/dm/interface/dm.c:559
#6 0x00007ffff71330b1 in DMDACreate3d (comm=1140850688, bx=DM_BOUNDARY_NONE, by=DM_BOUNDARY_NONE, bz=DM_BOUNDARY_NONE, stencil_type=DMDA_STENCIL_STAR, M=-128, N=-128, P=-128, m=-1, n=-1, p=-1, dof=10, s=1, lx=0x0, ly=0x0, lz=0x0, da=0x7fffffffdd28) at /home/jed/petsc/src/dm/impls/da/da3.c:1463
#7 0x0000000000400c96 in main (argc=4, argv=0x7fffffffde28) at /home/jed/petsc/src/dm/examples/tests/ex20.c:15
#0 PetscTrMallocDefault (a=41943040, lineno=1880, function=0x7ffff7b0449a <.L__func__.VecScatterCreate_PtoS> "VecScatterCreate_PtoS", filename=0x7ffff7b03f29 <.L.str> "/home/jed/petsc/src/vec/vec/utils/vpscat.c", result=0x516ba0) at /home/jed/petsc/src/sys/memory/mtr.c:228
#1 0x00007ffff6a9431f in VecScatterCreate_PtoS (nx=10649600, inidx=0x7fffeafb5650, ny=10649600, inidy=0x7fffe17e3650, xin=0x4be360, yin=0x4c1600, bs=1, ctx=0x514f70) at /home/jed/petsc/src/vec/vec/utils/vpscat.c:1880
#2 0x00007ffff6a5fa49 in VecScatterCreate (xin=0x4be360, ix=0x4cf6a0, yin=0x4c1600, iy=0x50ede0, newctx=0x7fffffffd850) at /home/jed/petsc/src/vec/vec/utils/vscat.c:1497
#3 0x00007ffff712ed23 in DMSetUp_DA_3D (da=0x4a88d0) at /home/jed/petsc/src/dm/impls/da/da3.c:1033
#4 0x00007ffff71a3ef5 in DMSetUp_DA (da=0x4a88d0) at /home/jed/petsc/src/dm/impls/da/dareg.c:27
#5 0x00007ffff736fd9f in DMSetUp (dm=0x4a88d0) at /home/jed/petsc/src/dm/interface/dm.c:559
#6 0x00007ffff71330b1 in DMDACreate3d (comm=1140850688, bx=DM_BOUNDARY_NONE, by=DM_BOUNDARY_NONE, bz=DM_BOUNDARY_NONE, stencil_type=DMDA_STENCIL_STAR, M=-128, N=-128, P=-128, m=-1, n=-1, p=-1, dof=10, s=1, lx=0x0, ly=0x0, lz=0x0, da=0x7fffffffdd28) at /home/jed/petsc/src/dm/impls/da/da3.c:1463
#7 0x0000000000400c96 in main (argc=4, argv=0x7fffffffde28) at /home/jed/petsc/src/dm/examples/tests/ex20.c:15
#0 PetscTrMallocDefault (a=42598400, lineno=47, function=0x7ffff7afd5d4 <.L__func__.ISGetIndices_Block> "ISGetIndices_Block", filename=0x7ffff7afd53b <.L.str> "/home/jed/petsc/src/vec/is/is/impls/block/block.c", result=0x7fffffffd4b8) at /home/jed/petsc/src/sys/memory/mtr.c:228
#1 0x00007ffff691fad2 in ISGetIndices_Block (in=0x50ede0, idx=0x7fffffffd5e0) at /home/jed/petsc/src/vec/is/is/impls/block/block.c:47
#2 0x00007ffff6936468 in ISGetIndices (is=0x50ede0, ptr=0x7fffffffd5e0) at /home/jed/petsc/src/vec/is/is/interface/index.c:382
#3 0x00007ffff68eb66a in ISLocalToGlobalMappingCreateIS (is=0x50ede0, mapping=0x4a97a0) at /home/jed/petsc/src/vec/is/utils/isltog.c:139
#4 0x00007ffff713216f in DMSetUp_DA_3D (da=0x4a88d0) at /home/jed/petsc/src/dm/impls/da/da3.c:1362
#5 0x00007ffff71a3ef5 in DMSetUp_DA (da=0x4a88d0) at /home/jed/petsc/src/dm/impls/da/dareg.c:27
#6 0x00007ffff736fd9f in DMSetUp (dm=0x4a88d0) at /home/jed/petsc/src/dm/interface/dm.c:559
#7 0x00007ffff71330b1 in DMDACreate3d (comm=1140850688, bx=DM_BOUNDARY_NONE, by=DM_BOUNDARY_NONE, bz=DM_BOUNDARY_NONE, stencil_type=DMDA_STENCIL_STAR, M=-128, N=-128, P=-128, m=-1, n=-1, p=-1, dof=10, s=1, lx=0x0, ly=0x0, lz=0x0, da=0x7fffffffdd28) at /home/jed/petsc/src/dm/impls/da/da3.c:1463
#8 0x0000000000400c96 in main (argc=4, argv=0x7fffffffde28) at /home/jed/petsc/src/dm/examples/tests/ex20.c:15
#0 PetscTrMallocDefault (a=83886080, lineno=197, function=0x7ffff7b0171f <.L__func__.VecCreate_MPI_Private> "VecCreate_MPI_Private", filename=0x7ffff7b01735 <.L.str> "/home/jed/petsc/src/vec/vec/impls/mpi/pbvec.c", result=0x513b60) at /home/jed/petsc/src/sys/memory/mtr.c:228
#1 0x00007ffff69e33d8 in VecCreate_MPI_Private (v=0x511c30, alloc=PETSC_TRUE, nghost=0, array=0x0) at /home/jed/petsc/src/vec/vec/impls/mpi/pbvec.c:197
#2 0x00007ffff69e3f5b in VecCreate_MPI (vv=0x511c30) at /home/jed/petsc/src/vec/vec/impls/mpi/pbvec.c:240
#3 0x00007ffff6a17cd1 in VecSetType (vec=0x511c30, method=0x7ffff7b01763 <.L.str6> "mpi") at /home/jed/petsc/src/vec/vec/interface/vecreg.c:53
#4 0x00007ffff69e4395 in VecCreate_Standard (v=0x511c30) at /home/jed/petsc/src/vec/vec/impls/mpi/pbvec.c:267
#5 0x00007ffff6a17cd1 in VecSetType (vec=0x511c30, method=0x4b9b00 "standard") at /home/jed/petsc/src/vec/vec/interface/vecreg.c:53
#6 0x00007ffff715ad11 in DMCreateGlobalVector_DA (da=0x4a88d0, g=0x7fffffffdd20) at /home/jed/petsc/src/dm/impls/da/dadist.c:43
#7 0x00007ffff7370f54 in DMCreateGlobalVector (dm=0x4a88d0, vec=0x7fffffffdd20) at /home/jed/petsc/src/dm/interface/dm.c:680
#8 0x0000000000400e25 in main (argc=4, argv=0x7fffffffde28) at /home/jed/petsc/src/dm/examples/tests/ex20.c:20
#0 PetscTrMallocDefault (a=83886080, lineno=197, function=0x7ffff7b0171f <.L__func__.VecCreate_MPI_Private> "VecCreate_MPI_Private", filename=0x7ffff7b01735 <.L.str> "/home/jed/petsc/src/vec/vec/impls/mpi/pbvec.c", result=0x4c9550) at /home/jed/petsc/src/sys/memory/mtr.c:228
#1 0x00007ffff69e33d8 in VecCreate_MPI_Private (v=0x4d0ad0, alloc=PETSC_TRUE, nghost=0, array=0x0) at /home/jed/petsc/src/vec/vec/impls/mpi/pbvec.c:197
#2 0x00007ffff69e3f5b in VecCreate_MPI (vv=0x4d0ad0) at /home/jed/petsc/src/vec/vec/impls/mpi/pbvec.c:240
#3 0x00007ffff6a17cd1 in VecSetType (vec=0x4d0ad0, method=0x7ffff7b01763 <.L.str6> "mpi") at /home/jed/petsc/src/vec/vec/interface/vecreg.c:53
#4 0x00007ffff69e4395 in VecCreate_Standard (v=0x4d0ad0) at /home/jed/petsc/src/vec/vec/impls/mpi/pbvec.c:267
#5 0x00007ffff6a17cd1 in VecSetType (vec=0x4d0ad0, method=0x4b9b00 "standard") at /home/jed/petsc/src/vec/vec/interface/vecreg.c:53
#6 0x00007ffff715ad11 in DMCreateGlobalVector_DA (da=0x4a88d0, g=0x7fffffffdd18) at /home/jed/petsc/src/dm/impls/da/dadist.c:43
#7 0x00007ffff7370f54 in DMCreateGlobalVector (dm=0x4a88d0, vec=0x7fffffffdd18) at /home/jed/petsc/src/dm/interface/dm.c:680
#8 0x0000000000400fb6 in main (argc=4, argv=0x7fffffffde28) at /home/jed/petsc/src/dm/examples/tests/ex20.c:25
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20140515/7bb67902/attachment.sig>
More information about the petsc-dev
mailing list