<div dir="ltr">I'll fix this in the release if no one has done it yet.<div>Hong<br><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Aug 10, 2015 at 4:27 PM, Barry Smith <span dir="ltr"><<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
Anthony,<br>
<br>
This crash is in PETSc code before it calls the SuperLU_DIST numeric factorization; likely we have a mistake such as assuming a process has at least one row of the matrix and need to fix it.<br>
<br>
Barry<br>
<br>
> 0x00007fe6ba609297 in MatLUFactorNumeric_SuperLU_DIST (F=0x1922b50,<br>
> A=0x14a6a70, info=0x19099f8)<br>
> at /home/anthony/LIB/petsc-3.6.1/src/mat/impls/aij/mpi/superlu_dist/superlu_dist.c:368<br>
> 368 colA_start = rstart + ajj[0]; /* the smallest global col index of A */<br>
<br>
<br>
<br>
> On Aug 10, 2015, at 3:50 PM, Anthony Haas <<a href="mailto:aph@email.arizona.edu">aph@email.arizona.edu</a>> wrote:<br>
><br>
> Hi Sherry,<br>
><br>
> I recently submitted a matrix for which I noticed that Superlu_dist was hanging when running on 4 processors with parallel symbolic factorization. I have been using the latest version of Superlu_dist and the code is not hanging anymore. However, I noticed that when running the same matrix (I have attached the matrix), the code crashes with the following SIGSEGV when running on 10 procs (with or without parallel symbolic factorization). It is probably overkill to run such a 'small' matrix on 10 procs but I thought that it might still be useful to report the problem?? See below for the error obtained when running with gdb and also a code snippet to reproduce the error.<br>
><br>
> Thanks,<br>
><br>
><br>
> Anthony<br>
><br>
><br>
><br>
> 1) ERROR in GDB<br>
><br>
> Program received signal SIGSEGV, Segmentation fault.<br>
> 0x00007fe6ba609297 in MatLUFactorNumeric_SuperLU_DIST (F=0x1922b50,<br>
> A=0x14a6a70, info=0x19099f8)<br>
> at /home/anthony/LIB/petsc-3.6.1/src/mat/impls/aij/mpi/superlu_dist/superlu_dist.c:368<br>
> 368 colA_start = rstart + ajj[0]; /* the smallest global col index of A */<br>
> (gdb)<br>
><br>
><br>
><br>
> 2) PORTION OF CODE TO REPRODUCE ERROR<br>
><br>
> Subroutine HowBigLUCanBe(rank)<br>
><br>
> IMPLICIT NONE<br>
><br>
> integer(i4b),intent(in) :: rank<br>
> integer(i4b) :: i,ct<br>
> real(dp) :: begin,endd<br>
> complex(dpc) :: sigma<br>
><br>
> PetscErrorCode ierr<br>
><br>
><br>
> if (rank==0) call cpu_time(begin)<br>
><br>
> if (rank==0) then<br>
> write(*,*)<br>
> write(*,*)'Testing How Big LU Can Be...'<br>
> write(*,*)'============================'<br>
> write(*,*)<br>
> endif<br>
><br>
> !sigma = (1.0d0,0.0d0)<br>
> !call MatAXPY(A,-sigma,B,DIFFERENT_NONZERO_PATTERN,ierr) ! on exit A = A-sigma*B<br>
><br>
> !call MatView(A,PETSC_VIEWER_STDOUT_WORLD,ierr)<br>
><br>
> !.....Write Matrix to ASCII and Binary Format<br>
> !call PetscViewerASCIIOpen(PETSC_COMM_WORLD,"Amat.m",viewer,ierr)<br>
> !call MatView(DXX,viewer,ierr)<br>
> !call PetscViewerDestroy(viewer,ierr)<br>
><br>
> !call PetscViewerBinaryOpen(PETSC_COMM_WORLD,"Amat_binary.m",FILE_MODE_WRITE,viewer,ierr)<br>
> !call MatView(A,viewer,ierr)<br>
> !call PetscViewerDestroy(viewer,ierr)<br>
><br>
> !...Load a Matrix in Binary Format<br>
> call PetscViewerBinaryOpen(PETSC_COMM_WORLD,"Amat_binary.m",FILE_MODE_READ,viewer,ierr)<br>
> call MatCreate(PETSC_COMM_WORLD,DLOAD,ierr)<br>
> call MatSetType(DLOAD,MATAIJ,ierr)<br>
> call MatLoad(DLOAD,viewer,ierr)<br>
> call PetscViewerDestroy(viewer,ierr)<br>
><br>
> !call MatView(DLOAD,PETSC_VIEWER_STDOUT_WORLD,ierr)<br>
><br>
><br>
> !.....Create Linear Solver Context<br>
> call KSPCreate(PETSC_COMM_WORLD,ksp,ierr)<br>
><br>
> !.....Set operators. Here the matrix that defines the linear system also serves as the preconditioning matrix.<br>
> !call KSPSetOperators(ksp,A,A,DIFFERENT_NONZERO_PATTERN,ierr) !aha commented and replaced by next line<br>
><br>
> !call KSPSetOperators(ksp,A,A,ierr) ! remember: here A = A-sigma*B<br>
> call KSPSetOperators(ksp,DLOAD,DLOAD,ierr) ! remember: here A = A-sigma*B<br>
><br>
> !.....Set Relative and Absolute Tolerances and Uses Default for Divergence Tol<br>
> tol = 1.e-10<br>
> call KSPSetTolerances(ksp,tol,tol,PETSC_DEFAULT_REAL,PETSC_DEFAULT_INTEGER,ierr)<br>
><br>
> !.....Set the Direct (LU) Solver<br>
> call KSPSetType(ksp,KSPPREONLY,ierr)<br>
> call KSPGetPC(ksp,pc,ierr)<br>
> call PCSetType(pc,PCLU,ierr)<br>
> call PCFactorSetMatSolverPackage(pc,MATSOLVERSUPERLU_DIST,ierr) ! MATSOLVERSUPERLU_DIST MATSOLVERMUMPS<br>
><br>
> !.....Create Right-Hand-Side Vector<br>
> !call MatCreateVecs(A,frhs,PETSC_NULL_OBJECT,ierr)<br>
> !call MatCreateVecs(A,sol,PETSC_NULL_OBJECT,ierr)<br>
><br>
> call MatCreateVecs(DLOAD,frhs,PETSC_NULL_OBJECT,ierr)<br>
> call MatCreateVecs(DLOAD,sol,PETSC_NULL_OBJECT,ierr)<br>
><br>
> call MatGetOwnershipRange(DLOAD,IstartA,IendA,ierr)!;CHKERRQ(ierr)<br>
><br>
> allocate(xwork1(IendA-IstartA))<br>
> allocate(loc(IendA-IstartA))<br>
><br>
> ct=0<br>
> do i=IstartA,IendA-1<br>
> ct=ct+1<br>
> loc(ct)=i<br>
> xwork1(ct)=(1.0d0,0.0d0)<br>
> enddo<br>
><br>
> call VecSetValues(frhs,IendA-IstartA,loc,xwork1,INSERT_VALUES,ierr)<br>
> call VecZeroEntries(sol,ierr)<br>
><br>
> deallocate(xwork1,loc)<br>
><br>
> !.....Assemble Vectors<br>
> call VecAssemblyBegin(frhs,ierr)<br>
> call VecAssemblyEnd(frhs,ierr)<br>
><br>
> !.....Solve the Linear System<br>
> call KSPSolve(ksp,frhs,sol,ierr)<br>
><br>
> !call VecView(sol,PETSC_VIEWER_STDOUT_WORLD,ierr)<br>
><br>
> if (rank==0) then<br>
> call cpu_time(endd)<br>
> write(*,*)<br>
> print '("Total time for HowBigLUCanBe = ",f21.3," seconds.")',endd-begin<br>
> endif<br>
><br>
> call SlepcFinalize(ierr)<br>
><br>
> STOP<br>
><br>
><br>
> end Subroutine HowBigLUCanBe<br>
><br>
> <<a href="http://Amat_binary.m.info" rel="noreferrer" target="_blank">Amat_binary.m.info</a>><br>
<br>
</blockquote></div><br></div></div></div>