[petsc-users] Error when switching from MPIAIJ to MPIBAIJ

[Matthew Knepley]

On Thu, Jun 9, 2011 at 7:57 AM, Henning Sauerland <uerland at gmail.com> wrote:

> I'd like to get an idea of the solver performance when using mpibaij. Using
> my existing code based on mpiaij I simply swapped the preallocation routine
> and switched to mpibaij using the runtime flag. I basically perform the
> following function calls:
>

1) You can call both preallocation routines, but note that the block version
takes smaller dnz/onz

2) If you want the matrix and vector to be divided the same way
automatically, give them the same blocksize

  Thanks,

     Matt


> MatCreate(PETSC_COMM_WORLD, &SystMat);
> MatSetSizes(SystMat, PETSC_DECIDE, PETSC_DECIDE, nDOFg, nDOFg);
> MatSetFromOptions(SystMat);
> MatMPIBAIJSetPreallocation(SystMat, 5, 30, PETSC_NULL,10, PETSC_NULL);
>
> VecCreate(PETSC_COMM_WORLD, &Rhs);
> VecSetSizes(Rhs, PETSC_DECIDE, nDOFg);
> VecSetFromOptions(Rhs);
>
> /* Filling SystMat and Rhs (currently using MatSetValues() ) */
> ...
>
> VecDuplicate(*Rhs, &Sol);
>
> KSPCreate(PETSC_COMM_WORLD, &ksp);
> KSPSetOperators(ksp, *SystMat, *SystMat, DIFFERENT_NONZERO_PATTERN);
> KSPSetFromOptions(ksp);
> KSPSolve(ksp, *Rhs, Sol);
>
>
> My runtime options are: -mat_type mpibaij -ksp_type lgmres -ksp_rtol 1.e-8
> -ksp_gmres_restart 200 -pc_type asm -log_summary
>
>
> This is what I get using 2 processors:
>
> [0]PETSC ERROR: --------------------- Error Message
> ------------------------------------
> [0]PETSC ERROR: Nonconforming object sizes!
> [0]PETSC ERROR: Vector wrong size 34178 for scatter 34180 (scatter forward
> and vector from != ctx from size)!
> [0]PETSC ERROR:
> ------------------------------------------------------------------------
> [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 5, Mon Sep 27 11:51:54
> CDT 2010
> [0]PETSC ERROR: See docs/changes/index.html for recent updates.
> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
> [0]PETSC ERROR: See docs/index.html for manual pages.
> [0]PETSC ERROR:
> ------------------------------------------------------------------------
> [0]PETSC ERROR: ... on a ... Thu Jun  9 14:01:13 2011
> [0]PETSC ERROR: Libraries linked from ...
> [0]PETSC ERROR: Configure run at Wed Oct  6 11:03:30 2010
> [0]PETSC ERROR: Configure options --with-cc=mpicc --download-scalapack=1
> --with-cxx=mpicxx --download-mumps=1 --download-ml=1 --download-parmetis=1
> --download-f-blas-lapack=1 PETSC_ARCH=linux-gnu-c-debug --download-blacs=1
> --with-fc=mpif90 --download-hypre=1
> [0]PETSC ERROR:
> ------------------------------------------------------------------------
> [0]PETSC ERROR: VecScatterBegin() line 1549 in src/vec/vec/utils/vscat.c
> [0]PETSC ERROR: PCApply_ASM() line 366 in src/ksp/pc/impls/asm/asm.c
> [0]PETSC ERROR: PCApply() line 357 in src/ksp/pc/interface/precon.c
> [0]PETSC ERROR: KSPInitialResidual() line 65 in
> src/ksp/ksp/interface/itres.c
> [0]PETSC ERROR: KSPSolve_LGMRES() line 376 in
> src/ksp/ksp/impls/gmres/lgmres/lgmres.c
> [0]PETSC ERROR: KSPSolve() line 396 in src/ksp/ksp/interface/itfunc.c
> [1]PETSC ERROR: --------------------- Error Message
> ------------------------------------
> [1]PETSC ERROR: Nonconforming object sizes!
> [1]PETSC ERROR: Vector wrong size 34177 for scatter 34175 (scatter forward
> and vector from != ctx from size)!
> [1]PETSC ERROR:
> ------------------------------------------------------------------------
> [1]PETSC ERROR: Petsc Release Version 3.1.0, Patch 5, Mon Sep 27 11:51:54
> CDT 2010
> [1]PETSC ERROR: See docs/changes/index.html for recent updates.
> [1]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
> [1]PETSC ERROR: See docs/index.html for manual pages.
> [1]PETSC ERROR:
> ------------------------------------------------------------------------
> [1]PETSC ERROR: ... on a ... Thu Jun  9 14:01:13 2011
> [1]PETSC ERROR: Libraries linked from ...
> [1]PETSC ERROR: Configure run at Wed Oct  6 11:03:30 2010
> [1]PETSC ERROR: Configure options --with-cc=mpicc --download-scalapack=1
> --with-cxx=mpicxx --download-mumps=1 --download-ml=1 --download-parmetis=1
> --download-f-blas-lapack=1 PETSC_ARCH=linux-gnu-c-debug --download-blacs=1
> --with-fc=mpif90 --download-hypre=1
> [1]PETSC ERROR:
> ------------------------------------------------------------------------
> [1]PETSC ERROR: VecScatterBegin() line 1549 in src/vec/vec/utils/vscat.c
> [1]PETSC ERROR: PCApply_ASM() line 366 in src/ksp/pc/impls/asm/asm.c
> [1]PETSC ERROR: PCApply() line 357 in src/ksp/pc/interface/precon.c
> [1]PETSC ERROR: KSPInitialResidual() line 65 in
> src/ksp/ksp/interface/itres.c
> [1]PETSC ERROR: KSPSolve_LGMRES() line 376 in
> src/ksp/ksp/impls/gmres/lgmres/lgmres.c
> [1]PETSC ERROR: KSPSolve() line 396 in src/ksp/ksp/interface/itfunc.c
>
> Is there a basic mistake in my procedure?
>
>
> Thanks
> Henning




-- 
What most experimenters take for granted before they begin their experiments
is infinitely more interesting than any results to which their experiments
lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20110609/33f64e5f/attachment.htm>