Performance issue with MatSetValues

[Barry Smith]

   This is just do to an incomplete implementation; a user kindly donated
the code to use but did not add support for periodicity because he did not
need it. If you look at src/dm/da/src/dainterp.c you will find the various
routines for setting up the interpolations. If you look at the code
for 3D_Q1 you will see how the periodic case is handled; you may be able
to modify the 3D_Q0 code to also handle the periodic case. This will then
resolve your difficulty.

Good luck,

    Barry


On Thu, 10 Aug 2006, Christophe Picard wrote:

>
> I think the memory is indeed not preallocated.
> Yes my problem is periodic, but if I try to use a periodic DA, the multigrid
> solver complains about it (see the end of the message). I believe the source
> of that problem is  DAGetInterpolation_3D_Q0().
>
> The problem I am trying to solve is a 3D Poisson equation with
> Neuman/Robin/Periodic boundary conditions. The boundary conditions are
> decided at runtime.
>
> If I use if-else-if statements to choose the DA here is the message
>
> Thanks,
> Christophe
>
> [0]PETSC ERROR: --------------------- Error Message
> ------------------------------------
> [0]PETSC ERROR: Invalid argument!
> [0]PETSC ERROR: Cannot handle periodic grid in x!
> [0]PETSC ERROR:
> ------------------------------------------------------------------------
> [0]PETSC ERROR: Petsc Development Version 2.3.1, Patch 14, Thu Jul  6 00:02:04
> CDT 2006 HG revision: 97334a27165ab031dddd67964dd7a97955e75d20
> [0]PETSC ERROR: See docs/changes/index.html for recent updates.
> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
> [0]PETSC ERROR: See docs/index.html for manual pages.
> [0]PETSC ERROR:
> ------------------------------------------------------------------------
> [0]PETSC ERROR: ././oceanus on a linux-gnu named tux194.llnl.gov by picard1
> Thu Aug 10 17:16:31 2006
> [0]PETSC ERROR: Libraries linked from
> /home/picard1/Tools/Petsc-Dev/lib/linux-gnu-c-real-debug
> [0]PETSC ERROR: Configure run at Thu Jul  6 16:05:13 2006
> [0]PETSC ERROR: Configure options --prefix=/home/picard1/Tools/Petsc-Dev
> --with-dynamic --with-shared --with-mpi=0 --with-superlu=1
> --download-superlu=ifneeded
> [0]PETSC ERROR:
> ------------------------------------------------------------------------
> [0]PETSC ERROR: DAGetInterpolation_3D_Q0() line 498 in
> src/dm/da/src/dainterp.c
> [0]PETSC ERROR: DAGetInterpolation() line 874 in src/dm/da/src/dainterp.c
> [0]PETSC ERROR: DMGetInterpolation() line 117 in src/dm/da/utils/dm.c
> [0]PETSC ERROR: DMMGSetUp() line 215 in src/snes/utils/damg.c
> [0]PETSC ERROR: DMMGSetDM() line 180 in src/snes/utils/damg.c
> [0]PETSC ERROR: --------------------- Error Message
> ------------------------------------
> [0]PETSC ERROR: Null argument, when expecting valid pointer!
> [0]PETSC ERROR: Null Object: Parameter # 1!
> [0]PETSC ERROR:
> ------------------------------------------------------------------------
> [0]PETSC ERROR: Petsc Development Version 2.3.1, Patch 14, Thu Jul  6 00:02:04
> CDT 2006 HG revision: 97334a27165ab031dddd67964dd7a97955e75d20
> [0]PETSC ERROR: See docs/changes/index.html for recent updates.
> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
> [0]PETSC ERROR: See docs/index.html for manual pages.
> [0]PETSC ERROR:
> ------------------------------------------------------------------------
> [0]PETSC ERROR: ././oceanus on a linux-gnu named tux194.llnl.gov by picard1
> Thu Aug 10 17:16:31 2006
> [0]PETSC ERROR: Libraries linked from
> /home/picard1/Tools/Petsc-Dev/lib/linux-gnu-c-real-debug
> [0]PETSC ERROR: Configure run at Thu Jul  6 16:05:13 2006
> [0]PETSC ERROR: Configure options --prefix=/home/picard1/Tools/Petsc-Dev
> --with-dynamic --with-shared --with-mpi=0 --with-superlu=1
> --download-superlu=ifneeded
> [0]PETSC ERROR:
> ------------------------------------------------------------------------
> [0]PETSC ERROR: PetscObjectReference() line 106 in src/sys/objects/inherit.c
> [0]PETSC ERROR: MGSetInterpolate() line 136 in src/ksp/pc/impls/mg/mgfunc.c
> [0]PETSC ERROR: DMMGSetUpLevel() line 385 in src/snes/utils/damg.c
> [0]PETSC ERROR: DMMGSetKSP() line 452 in src/snes/utils/damg.c
>
>
>
> On Thursday 10 August 2006 04:53 pm, Matthew Knepley wrote:
>> It sounds like you are inserting values which were not preallocating. To
>> determine for sure, we would need to know more about the code. However,
>> if you have a periodic problem, why not use a periodic DA?
>>
>>    Matt
>>
>> On 8/10/06, Christophe Picard <picard2 at llnl.gov> wrote:
>>> Hi,
>>>
>>> I have a performance issue while trying to insert values in the a matrix.
>>> I am using DMMG solver for cell-centered scheme in 3D from the
>>> petsc-snapshot to solve a Poisson equations. Inserting coefficients in
>>> the matrix for dirichlet or neumann boundary conditions, the insertion is
>>> instantaneous. But is I want to insert coefficients for periodic boundary
>>> conditions, I can notice a huge slow down in the insertion process (not
>>> in the resolution though). The smallest size I can notice the performance
>>> drop is 32*32*32.
>>>
>>> Is there any way to improve this?
>>>
>>>
>>> col.i  = row.i+(mx-1);
>>> col.j  = row.j;
>>> col.k = row.k;
>>>
>>> v[0] =  1;
>>>
>>> MatSetValuesStencil(*A,1,&row,1,&col,v,ADD_VALUES);
>>>
>>>
>>>
>>> Thans,
>>>
>>> Christophe
>
>