[petsc-users] Error in creation of MPISBAIJ matrix with DMCreateMatrix
Xiao, Jianjun (IKET)
jianjun.xiao at kit.edu
Wed Jan 22 05:15:34 CST 2014
Dear Barry,
I modified ex44f.F90, and ran the case on my 64-processors cluster. Please find the file in the attachment.
I tried various number of processors. It seems that for number of processors: 1, 2, 3, 4, 5, 6, 7, 8, 16, 32 and 64 it works fine. For numbers such as 20, 30, 31, 60, 61 and 62, I got the error below. This time, even for MATMPIBAIJ I got error. I did not try all the numbers between 1-64. If you need more information, please let me know.
Thank you for your help.
Best regards
JJ
For MATMPISBAIJ, I got the error like this:
[55]PETSC ERROR: [55] MatAssemblyBegin_MPISBAIJ line 483 src/mat/impls/sbaij/mpi/mpisbaij.c
[55]PETSC ERROR: [55] MatAssemblyBegin line 4865 src/mat/interface/matrix.c
[55]PETSC ERROR: [55] DMCreateMatrix_DA_3d_MPISBAIJ line 1694 src/dm/impls/da/fdda.c
[55]PETSC ERROR: [55] DMCreateMatrix_DA line 626 src/dm/impls/da/fdda.c
[55]PETSC ERROR: [55] DMCreateMatrix line 1002 src/dm/interface/dm.c
[55]PETSC ERROR: --------------------- Error Message ------------------------------------
[55]PETSC ERROR: Signal received!
[55]PETSC ERROR: ------------------------------------------------------------------------
[55]PETSC ERROR: Petsc Development GIT revision: f7404d5510646a3c64be49fff6ce547efef07b3d GIT Date: 2013-11-27 00:12:54 +0100
[55]PETSC ERROR: See docs/changes/index.html for recent updates.
[55]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
[55]PETSC ERROR: See docs/index.html for manual pages.
[55]PETSC ERROR: ------------------------------------------------------------------------
[55]PETSC ERROR: ./ex44f on a linux-gnu named cluster07 by xiao Wed Jan 22 11:54:32 2014
[55]PETSC ERROR: Libraries linked from /home/xiao/Local/petsc-dev-debug/linux-gnu/lib
[55]PETSC ERROR: Configure run at Tue Jan 14 15:27:57 2014
[55]PETSC ERROR: Configure options --with-cc=mpicc --with-fc=mpif90 --with-cxx=mpicxx --with-debugging=1
[55]PETSC ERROR: ------------------------------------------------------------------------
[55]PETSC ERROR: User provided function() line 0 in unknown file
For MATMPIBAIJ, I got the error like this:
[44]PETSC ERROR: --------------------- Error Message ------------------------------------
[44]PETSC ERROR: Argument out of range!
[44]PETSC ERROR: Trying to set preallocation for row 6678 less than first local row 6714!
[44]PETSC ERROR: ------------------------------------------------------------------------
[44]PETSC ERROR: Petsc Development GIT revision: f7404d5510646a3c64be49fff6ce547efef07b3d GIT Date: 2013-11-27 00:12:54 +0100
[44]PETSC ERROR: See docs/changes/index.html for recent updates.
[44]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
[44]PETSC ERROR: See docs/index.html for manual pages.
[44]PETSC ERROR: ------------------------------------------------------------------------
[44]PETSC ERROR: ./ex44f on a linux-gnu named cluster06 by xiao Wed Jan 22 11:52:08 2014
[44]PETSC ERROR: Libraries linked from /home/xiao/Local/petsc-dev-debug/linux-gnu/lib
[44]PETSC ERROR: Configure run at Tue Jan 14 15:27:57 2014
[44]PETSC ERROR: Configure options --with-cc=mpicc --with-fc=mpif90 --with-cxx=mpicxx --with-debugging=1
[44]PETSC ERROR: ------------------------------------------------------------------------
[44]PETSC ERROR: DMCreateMatrix_DA_3d_MPIBAIJ() line 1508 in src/dm/impls/da/fdda.c
[44]PETSC ERROR: DMCreateMatrix_DA() line 771 in src/dm/impls/da/fdda.c
[44]PETSC ERROR: DMCreateMatrix() line 1007 in src/dm/interface/dm.c
[44]PETSC ERROR: --------------------- Error Message ------------------------------------
[44]PETSC ERROR: Corrupt argument:
see http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind!
[44]PETSC ERROR: Invalid Pointer to Object: Parameter # 1!
[44]PETSC ERROR: ------------------------------------------------------------------------
[44]PETSC ERROR: Petsc Development GIT revision: f7404d5510646a3c64be49fff6ce547efef07b3d GIT Date: 2013-11-27 00:12:54 +0100
[44]PETSC ERROR: See docs/changes/index.html for recent updates.
[44]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
[44]PETSC ERROR: See docs/index.html for manual pages.
[44]PETSC ERROR: ------------------------------------------------------------------------
[44]PETSC ERROR: ./ex44f on a linux-gnu named cluster06 by xiao Wed Jan 22 11:52:08 2014
[44]PETSC ERROR: Libraries linked from /home/xiao/Local/petsc-dev-debug/linux-gnu/lib
[44]PETSC ERROR: Configure run at Tue Jan 14 15:27:57 2014
[44]PETSC ERROR: Configure options --with-cc=mpicc --with-fc=mpif90 --with-cxx=mpicxx --with-debugging=1
[44]PETSC ERROR: ------------------------------------------------------------------------
[44]PETSC ERROR: MatDestroy() line 1029 in src/mat/interface/matrix.c
________________________________________
From: Barry Smith [bsmith at mcs.anl.gov]
Sent: Monday, January 20, 2014 8:37 PM
To: Xiao, Jianjun (IKET)
Cc: petsc-users at mcs.anl.gov; jedbrown at mcs.anl.gov
Subject: Re: [petsc-users] Error in creation of MPISBAIJ matrix with DMCreateMatrix
Thanks for reporting the problem. This is our error. Could you please send us the code that generates the error so we can reproduce the problem, determine the cause and fix it.
Barry
We need to know the exact values of imax,jmax, kmax etc to reproduce the problem.
On Jan 20, 2014, at 9:33 AM, Xiao, Jianjun (IKET) <jianjun.xiao at kit.edu> wrote:
> Dear developers,
>
> I am using petsc-dev. I tried to create a MPISBAIJ matrix as shown below, and it seems that the matrix creation is sensitive to the number of processors.
>
> CALL DMDACreate3d(PETSC_COMM_WORLD,DMDA_BOUNDARY_NONE, &
> & DMDA_BOUNDARY_NONE,DMDA_BOUNDARY_NONE, &
> & DMDA_STENCIL_BOX,-imax,-jmax,-kmax,PETSC_DECIDE,PETSC_DECIDE,&
> & PETSC_DECIDE,1,1,PETSC_NULL_INTEGER,PETSC_NULL_INTEGER, &
> & PETSC_NULL_INTEGER,da,ierr)
>
> CALL DMSetMatType(da,MATMPISBAIJ,ierr)
> CALL DMCreateMatrix(da,mat,ierr)
>
> A cluster with 64 processors was used for the testing.
>
> When the number of procssors are 1,2,3,4,5,6,7,8,16,32 and 64, the code always works quite well.
>
> For some other numbers, the code works not so stable. Sometimes, the matrix was created successfully. Sometimes, it failed.
>
> When the number of procssors are 20, 33, 63 or some relatively bigger numbers , the code always got the error below.
>
>
> mpirun: [25]PETSC ERROR: --------------------- Error Message ------------------------------------
> mpirun: [25]PETSC ERROR: Argument out of range!
> mpirun: [25]PETSC ERROR: New nonzero at (198,7038) caused a malloc!
> mpirun: [25]PETSC ERROR: ------------------------------------------------------------------------
> mpirun: [25]PETSC ERROR: Petsc Development GIT revision: f7404d5510646a3c64be49fff6ce547efef07b3d GIT Date: 2013-11-27 00:12:54 +0100
> mpirun: [25]PETSC ERROR: See docs/changes/index.html for recent updates.
> mpirun: [25]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
> mpirun: [25]PETSC ERROR: See docs/index.html for manual pages.
> mpirun: [25]PETSC ERROR: Configure options --with-cc=mpicc --with-fc=mpif90 --with-cxx=mpicxx --with-debugging=1
> mpirun: [25]PETSC ERROR: ------------------------------------------------------------------------
> mpirun: [25]PETSC ERROR: MatSetValuesBlocked_SeqBAIJ() line 1836 in src/mat/impls/baij/seq/baij.c
> mpirun: [25]PETSC ERROR: MatSetValuesBlocked_MPISBAIJ() line 339 in src/mat/impls/sbaij/mpi/mpisbaij.c
> mpirun: [25]PETSC ERROR: MatSetValuesBlocked() line 1658 in src/mat/interface/matrix.c
> mpirun: [25]PETSC ERROR: DMCreateMatrix_DA_3d_MPISBAIJ() line 1780 in src/dm/impls/da/fdda.c
> mpirun: [25]PETSC ERROR: DMCreateMatrix_DA() line 777 in src/dm/impls/da/fdda.c
> mpirun: [25]PETSC ERROR: DMCreateMatrix() line 1007 in src/dm/interface/dm.c
>
> Then I changed the matrix format to MATMPIBAIJ. DMCreateMatrix worked fine for any number of processors.
>
> CALL DMSetMatType(da,MATMPIBAIJ,ierr)
> CALL DMCreateMatrix(da,gfmat,ierr)
>
> Could you please let me know how can I fix this problem? If you need more information, please let me know. Thank you.
>
> JJ
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ex44f.F90
Type: text/x-fortran
Size: 3322 bytes
Desc: ex44f.F90
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20140122/f55ca1ff/attachment.bin>
More information about the petsc-users
mailing list