<html><head><meta http-equiv="Content-Type" content="text/html charset=iso-8859-1"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">You can try running a problem that finishes with<div><br></div><div><div>-malloc</div><div>-malloc_debug</div><div>-malloc_dump</div><div><br></div><div>to see if there are any memory issues. It looks like you are completely destroying the solver object each time and starting over. Valgrind would be very useful here because it looks like memory is getting corrupted if the same code runs many times.</div><div><br></div><div><br></div><div><div>On Oct 19, 2013, at 3:39 PM, Einar Sørheim <<a href="mailto:einar.sorheim@gmail.com">einar.sorheim@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div dir="ltr">yes, I call the given code many times, what i have seen so far is that changing e.g. the order of destruction statements changes the time of failure,I will try to debug to get more details</div><div class="gmail_extra">
<br><br><div class="gmail_quote">2013/10/18 Mark F. Adams <span dir="ltr"><<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">I would guess there is some memory corruption or leak someplace. Try running with -on_error_attach_debugger, and try to get some more information. Do you just repeat a call to the code below many times? Is this deterministic, does it fail on the same solve each time? <div>
<br></div><div><br><div><div><div class="h5"><div>On Oct 18, 2013, at 9:30 AM, Einar Sørheim <<a href="mailto:einar.sorheim@gmail.com" target="_blank">einar.sorheim@gmail.com</a>> wrote:</div><br></div></div><blockquote type="cite">
<div><div class="h5"><div dir="ltr">Uncommenting <span style="font-family:arial,sans-serif;font-size:13px"> KSPSetFromOptions solved my intial problem, however I got a new problem that occurs after many timesteps, in our code we use petsc gamg as a solver for several problems(e.g. thermal and mechanical) at each time step, I have a suspicion it might have something to do with cleanup after solving, the rror appears after many calls to the petsc gamg solver interface, the error message is the following:</span><div>
<font face="arial, sans-serif"><br></font><div>[0]PETSC ERROR: ------------------------------------------------------------------------</div><div>[0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range</div>
<div>[0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger</div><div>[0]PETSC ERROR: or see <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[0]PETSC" target="_blank">http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[0]PETSC</a> ERROR: or try <a href="http://valgrind.org/" target="_blank">http://valgrind.org</a> on GNU/linux and Apple Mac OS X to find memory corruption errors</div>
<div>[0]PETSC ERROR: likely location of problem given in stack below</div><div>[0]PETSC ERROR: --------------------- Stack Frames ------------------------------------</div><div>[0]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,</div>
<div>[0]PETSC ERROR: INSTEAD the line number of the start of the function</div><div>[0]PETSC ERROR: is given.</div><div>[0]PETSC ERROR: [0] MatSetNearNullSpace line 7580 src/mat/interface/D:\PROGRA~1\PETSC-~1.3\src\mat\INTERF~1\matrix.c</div>
<div>[0]PETSC ERROR: --------------------- Error Message ------------------------------------</div><div>[0]PETSC ERROR: Signal received!</div><div>[0]PETSC ERROR: ------------------------------------------------------------------------</div>
<div>[0]PETSC ERROR: Petsc Release Version 3.4.3, Oct, 15, 2013 </div><div>[0]PETSC ERROR: See docs/changes/index.html for recent updates.</div><div>[0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.</div>
<div>[0]PETSC ERROR: See docs/index.html for manual pages.</div><div>[0]PETSC ERROR: ------------------------------------------------------------------------</div><div>[0]PETSC ERROR: D:\Programming\gits\Alsim\SharedModules - Copy\Source\Release\Alsim_nypetsc.exe on a arch-mswin-c-debug named CMP3 by admeinar Fri Oct 18 12:26:04 2013</div>
<div>[0]PETSC ERROR: Libraries linked from /cygdrive/d/Programming/petsc-3.4.3/arch-mswin-c-debug/lib</div><div>[0]PETSC ERROR: Configure run at Thu Oct 17 08:23:18 2013</div><div>[0]PETSC ERROR: Configure options --with-debugging=1 --with-openmp=yes --with-pthread=no --with-cc="win32fe icl" --with-fc="win32fe ifort" --with-cxx="win32fe icl" --with-blas-lapack-dir="C:/Program Files (x86)/Intel/Composer XE 2013/mkl/lib/intel64" --download-superlu --with-sowing=0 --with-c2html=0 --with-mpi-dir="C:/Program Files (x86)/Intel/MPI/<a href="http://4.1.0.028/em64t" target="_blank">4.1.0.028/em64t</a>" --useThreads=0 --useThreads=0</div>
<div><span style="font-family:arial,sans-serif;font-size:13px"><br></span></div><div><span style="font-family:arial,sans-serif;font-size:13px">The source code for the petsc interface: </span></div><div><span style="font-family:arial,sans-serif;font-size:13px"><br>
</span></div><div><div><font face="arial, sans-serif"> subroutine petscsolver(i_nodes,r_elements_,i_rowstart_,i_columnindex_,r_x_,r_b_,i_Nnz,i_Precond, i_Solver</font></div><div><font face="arial, sans-serif"> $ ,coords_,i_nodedof)</font></div>
<div><font face="arial, sans-serif">#include <finclude/petscsys.h></font></div><div><font face="arial, sans-serif">#include <finclude/petscvec.h></font></div><div><font face="arial, sans-serif">#include <finclude/petscmat.h></font></div>
<div><font face="arial, sans-serif">#include <finclude/petscpc.h></font></div><div><font face="arial, sans-serif">#include <finclude/petscksp.h></font></div><div><font face="arial, sans-serif">#include <finclude/petscviewer.h></font></div>
<div><font face="arial, sans-serif"><br></font></div><div><font face="arial, sans-serif"> integer i_nodes, i_Nnz, i_Precond, i_solver, i_nodedof</font></div><div><font face="arial, sans-serif"> integer i_rowstart_(*),i_columnindex_(*)</font></div>
<div><font face="arial, sans-serif"> double precision r_elements_(*),r_x_(*), r_b_(*),coords_(*)</font></div><div><font face="arial, sans-serif"> integer, allocatable :: i_indices_(:)</font></div><div><font face="arial, sans-serif">! Variables:</font></div>
<div><font face="arial, sans-serif">!</font></div><div><font face="arial, sans-serif">! A - matrix that defines linear system</font></div><div><font face="arial, sans-serif">! ksp - KSP context</font></div><div>
<font face="arial, sans-serif">! ksp - KSP context</font></div><div><font face="arial, sans-serif">! x, b, u - approx solution, RHS, exact solution vectors</font></div><div><font face="arial, sans-serif">!</font></div>
<div><font face="arial, sans-serif">! implicit none</font></div><div><font face="arial, sans-serif"> Vec x,u,b,vec_coords</font></div><div><font face="arial, sans-serif"> PC pc</font></div><div><font face="arial, sans-serif"> Mat A,F,Matnull</font></div>
<div><font face="arial, sans-serif"> KSP ksp</font></div><div><font face="arial, sans-serif"> PetscInt i,j,II,JJ,m,n,i1,imaxiter</font></div><div><font face="arial, sans-serif"> PetscInt Istart,Iend</font></div>
<div><font face="arial, sans-serif"> PetscInt nsteps,one, neg_one</font></div><div><font face="arial, sans-serif"> PetscErrorCode ierr</font></div><div><font face="arial, sans-serif"> PetscBool flg</font></div>
<div><font face="arial, sans-serif"> PetscScalar v,x_array_(1), norm, tol, t0,t1</font></div><div><font face="arial, sans-serif"> PetscOffset i_x</font></div><div><font face="arial, sans-serif"> PetscViewer viewer</font></div>
<div><font face="arial, sans-serif"><br></font></div><div><br></div><div><font face="arial, sans-serif"> call PetscTime(t0,ierr)</font></div><div><font face="arial, sans-serif"> write(*,*) "Enter petsc"</font></div>
<div><br></div><div><font face="arial, sans-serif"> n = 1</font></div><div><font face="arial, sans-serif"> nsteps = 1</font></div><div><font face="arial, sans-serif"> one = 1</font></div><div><font face="arial, sans-serif"> neg_one = 1</font></div>
<div><font face="arial, sans-serif"> i1 = 1</font></div><div><font face="arial, sans-serif"> imaxiter = 1200</font></div><div><br></div><div><br></div><div><font face="arial, sans-serif"> call MatCreate(PETSC_COMM_WORLD,A,ierr)</font></div>
<div><font face="arial, sans-serif"> call MatSetSizes(A,PETSC_DECIDE,PETSC_DECIDE,i_nodes,i_nodes,ierr)</font></div><div><font face="arial, sans-serif"> call MatSetType(A, MATSEQAIJ,ierr)</font></div><div><font face="arial, sans-serif"> call MatSetUp(A,ierr)</font></div>
<div><font face="arial, sans-serif"> call MatSeqAIJSetPreallocation(A,100,PETSC_NULL_INTEGER,ierr)</font></div><div><font face="arial, sans-serif">! The matrix is partitioned by contiguous chunks of rows across the</font></div>
<div><font face="arial, sans-serif">! processors. Determine which rows of the matrix are locally owned. </font></div><div><font face="arial, sans-serif"><br></font></div><div><font face="arial, sans-serif"> call MatGetOwnershipRange(A,Istart,Iend,ierr)</font></div>
<div><font face="arial, sans-serif"><br></font></div><div><br></div><div><font face="arial, sans-serif"> write(*,*) "Start of matrix fill", i_nodes, Istart, Iend</font></div><div><font face="arial, sans-serif"> do i=1,i_nodes</font></div>
<div><font face="arial, sans-serif">! write(*,*) "Start of loop",i !!, i_indices_(i+1)</font></div><div><font face="arial, sans-serif">! i_indices_(i+1) = i</font></div><div><font face="arial, sans-serif">! write(*,*) "Start of loop2"</font></div>
<div><font face="arial, sans-serif"> do j=i_rowstart_(i),i_rowstart_(i+1)-1</font></div><div><font face="arial, sans-serif">! write(*,*) "Start of loop 3"</font></div><div><font face="arial, sans-serif"> v = r_elements_(j)</font></div>
<div><font face="arial, sans-serif">! write(*,*) i1,i,i1,j-1,v</font></div><div><font face="arial, sans-serif"> call MatSetValues(A,i1,i-1,i1,i_columnindex_(j)-1,v,INSERT_VALUES,ierr)</font></div><div>
<font face="arial, sans-serif"> if (ierr.gt.0) stop</font></div><div><font face="arial, sans-serif"> end do</font></div><div><font face="arial, sans-serif"> end do</font></div><div><font face="arial, sans-serif">! write(*,*) "End of matrix fill"</font></div>
<div><font face="arial, sans-serif"><br></font></div><div><font face="arial, sans-serif">! Assemble matrix, using the 2-step process:</font></div><div><font face="arial, sans-serif">! MatAssemblyBegin(), MatAssemblyEnd()</font></div>
<div><font face="arial, sans-serif">! Computations can be done while messages are in transition</font></div><div><font face="arial, sans-serif">! by placing code between these two statements.</font></div><div><font face="arial, sans-serif"><br>
</font></div><div><font face="arial, sans-serif"> call MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY,ierr)</font></div><div><font face="arial, sans-serif"> call MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY,ierr)</font></div><div>
<font face="arial, sans-serif"><br></font></div><div><font face="arial, sans-serif">! Create parallel vectors.</font></div><div><font face="arial, sans-serif">! - When using VecCreate(), the parallel partitioning of the vector</font></div>
<div><font face="arial, sans-serif">! is determined by PETSc at runtime.</font></div><div><font face="arial, sans-serif">! - Note: We form 1 vector from scratch and then duplicate as needed.</font></div><div><font face="arial, sans-serif">! write(*,*) "Start of vector fill"</font></div>
<div><font face="arial, sans-serif"> call VecCreate(PETSC_COMM_WORLD,u,ierr)</font></div><div><font face="arial, sans-serif"> call VecSetType(u, VECSEQ,ierr)</font></div><div><font face="arial, sans-serif"> call VecSetSizes(u,PETSC_DECIDE,i_nodes,ierr)</font></div>
<div><font face="arial, sans-serif">! call VecSetFromOptions(u,ierr) </font></div><div><font face="arial, sans-serif"> call VecDuplicate(u,b,ierr)</font></div><div><font face="arial, sans-serif"> call VecDuplicate(b,x,ierr)</font></div>
<div><font face="arial, sans-serif"> do i=1,i_nodes</font></div><div><font face="arial, sans-serif"> call VecSetValues(x,i1,i-1,r_x_(i),INSERT_VALUES,ierr)</font></div><div><font face="arial, sans-serif"> call VecSetValues(b,i1,i-1,r_b_(i),INSERT_VALUES,ierr)</font></div>
<div><font face="arial, sans-serif"> enddo</font></div><div><font face="arial, sans-serif">! Assemble vector</font></div><div><font face="arial, sans-serif"><br></font></div><div><font face="arial, sans-serif"> call VecAssemblyBegin(x,ierr)</font></div>
<div><font face="arial, sans-serif"> call VecAssemblyEnd(x,ierr)</font></div><div><font face="arial, sans-serif"> call VecAssemblyBegin(b,ierr)</font></div><div><font face="arial, sans-serif"> call VecAssemblyEnd(b,ierr)</font></div>
<div><font face="arial, sans-serif"> call PetscTime(t1,ierr)</font></div><div><font face="arial, sans-serif">! write(*,*) "End of vec fill time spent :", t1-t0</font></div><div><font face="arial, sans-serif">! Create linear solver context</font></div>
<div><font face="arial, sans-serif"><br></font></div><div><font face="arial, sans-serif"> call KSPCreate(PETSC_COMM_WORLD,ksp,ierr)</font></div><div><font face="arial, sans-serif"><br></font></div><div><br></div><div>
<font face="arial, sans-serif">! Set operators. Here the matrix that defines the linear system</font></div><div><font face="arial, sans-serif">! also serves as the preconditioning matrix.</font></div><div><font face="arial, sans-serif"><br>
</font></div><div><font face="arial, sans-serif"> call KSPSetOperators(ksp,A,A,DIFFERENT_NONZERO_PATTERN,ierr)</font></div><div><font face="arial, sans-serif"><br></font></div><div><br></div><div><font face="arial, sans-serif"> call KSPGetPC(ksp,pc,ierr)</font></div>
<div><font face="arial, sans-serif"> tol = 1.e-20</font></div><div><font face="arial, sans-serif"> call KSPSetTolerances(ksp,tol,PETSC_DEFAULT_DOUBLE_PRECISION,PETSC_DEFAULT_DOUBLE_PRECISION,imaxiter,ierr)</font></div>
<div><font face="arial, sans-serif"> select case (i_Precond)</font></div><div><font face="arial, sans-serif"> case ( 1 )</font></div><div><font face="arial, sans-serif"> call PCSetType(pc,PCJACOBI,ierr)</font></div>
<div><font face="arial, sans-serif"> case ( 2 )</font></div><div><font face="arial, sans-serif"> call PCSetType(pc,PCILU,ierr)</font></div><div><font face="arial, sans-serif"> call PCFactorSetMatSolverPackage(pc,MATSOLVERSUPERLU,ierr);</font></div>
<div><font face="arial, sans-serif"> call PCFactorSetUpMatSolverPackage(pc,ierr); </font></div><div><font face="arial, sans-serif"> call PCFactorGetMatrix(pc,F);</font></div><div><font face="arial, sans-serif"> call MatSuperluSetILUDropTol(F,1.e-4);</font></div>
<div><font face="arial, sans-serif"> case ( 3 )</font></div><div><font face="arial, sans-serif"> call PCSetType(pc,PCGAMG,ierr)</font></div><div><font face="arial, sans-serif">! call PCGAMGInitializePackage()</font></div>
<div><font face="arial, sans-serif">! call PCCreate_GAMG(pc,ierr)</font></div><div><font face="arial, sans-serif"> case DEFAULT</font></div><div><font face="arial, sans-serif"> call PCSetType(pc,PCJACOBI,ierr)</font></div>
<div><font face="arial, sans-serif"> end select</font></div><div><font face="arial, sans-serif"><br></font></div><div><font face="arial, sans-serif">! call PCSetType(pc,PCJACOBI,ierr)</font></div><div><font face="arial, sans-serif"> select case (i_solver)</font></div>
<div><font face="arial, sans-serif"> case ( 0 )</font></div><div><font face="arial, sans-serif"> call KSPSetType(ksp,KSPCG,ierr)</font></div><div><font face="arial, sans-serif"> case DEFAULT</font></div>
<div><font face="arial, sans-serif"> call KSPSetType(ksp,KSPCG,ierr)</font></div><div><font face="arial, sans-serif"> end select</font></div><div><font face="arial, sans-serif"> call KSPCGSetType(ksp,KSP_CG_SYMMETRIC,ierr)</font></div>
<div><font face="arial, sans-serif"> if (i_nodedof==3) then</font></div><div><font face="arial, sans-serif"> write(*,*) "Set coord, number of nodes is:", i_nodes/i_nodedof</font></div><div><font face="arial, sans-serif"> call VecCreateSeqWithArray(MPI_COMM_SELF,3,i_nodes,coords_,vec_coords,ierr)</font></div>
<div><font face="arial, sans-serif"> call MatNullSpaceCreateRigidBody(vec_coords,Matnull,ierr)</font></div><div><font face="arial, sans-serif"> call MatSetNearNullSpace(A,Matnull,ierr)</font></div><div><font face="arial, sans-serif"> call MatNullSpaceDestroy(Matnull,ierr)</font></div>
<div><br></div><div><font face="arial, sans-serif"> write(*,*) "Finish setting null space ierr =", ierr</font></div><div><font face="arial, sans-serif">! call PCSetCoordinates( pc, 3,i_nodes/i_nodedof, coords_, ierr )</font></div>
<div><br></div><div><font face="arial, sans-serif"> end if</font></div><div><font face="arial, sans-serif">! call KSPMonitorSet(ksp, KSPMonitorDefault, PETSC_NULL, PETSC_NULL,ierr)</font></div><div><font face="arial, sans-serif">! Set runtime options, e.g.,</font></div>
<div><font face="arial, sans-serif">! -ksp_type <type> -pc_type <type> -ksp_monitor -ksp_rtol <rtol></font></div><div><font face="arial, sans-serif">! These options will override those specified above as long as</font></div>
<div><font face="arial, sans-serif">! KSPSetFromOptions() is called _after_ any other customization</font></div><div><font face="arial, sans-serif">! routines.</font></div><div><font face="arial, sans-serif"><br></font></div>
<div><font face="arial, sans-serif"> call KSPSetFromOptions(ksp,ierr)</font></div><div><font face="arial, sans-serif"> call KSPSetInitialGuessNonzero(ksp,PETSC_TRUE,ierr)</font></div><div><font face="arial, sans-serif"> call KSPSetUp(ksp,ierr)</font></div>
<div><font face="arial, sans-serif"> call PetscTime(t0,ierr)</font></div><div><font face="arial, sans-serif"> write(*,*) "Time setup", t0-t1</font></div><div><font face="arial, sans-serif">! - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - </font></div>
<div><font face="arial, sans-serif">! Solve the linear system</font></div><div><font face="arial, sans-serif">! - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - </font></div><div>
<font face="arial, sans-serif"><br></font></div><div><font face="arial, sans-serif"> call KSPSolve(ksp,b,x,ierr)</font></div><div><font face="arial, sans-serif"> call PetscTime(t1,ierr)</font></div><div><font face="arial, sans-serif"> write(*,*) "end solve, time used:",t1-t0</font></div>
<div><font face="arial, sans-serif"> call KSPGetSolution(ksp,x,ierr)</font></div><div><font face="arial, sans-serif"> call VecGetArray(x,x_array_,i_x,ierr)</font></div><div><font face="arial, sans-serif"> do i=1,i_nodes</font></div>
<div><font face="arial, sans-serif"> r_x_(i) = x_array_(i_x + i)</font></div><div><font face="arial, sans-serif"> enddo</font></div><div><font face="arial, sans-serif"> call VecRestoreArray(x,x_array_,i_x,ierr)</font></div>
<div><br></div><div><font face="arial, sans-serif"><br></font></div><div><font face="arial, sans-serif">! Free work space. All PETSc objects should be destroyed when they</font></div><div><font face="arial, sans-serif">! are no longer needed.</font></div>
<div><font face="arial, sans-serif">! call PCDestroy(pc,ierr)</font></div><div><font face="arial, sans-serif"><br></font></div><div><font face="arial, sans-serif"> call VecDestroy(u,ierr)</font></div><div><font face="arial, sans-serif"> call VecDestroy(x,ierr)</font></div>
<div><font face="arial, sans-serif"> call VecDestroy(b,ierr)</font></div><div><font face="arial, sans-serif"> call MatDestroy(A,ierr)</font></div><div><font face="arial, sans-serif"> if (i_nodedof==3) then</font></div>
<div><font face="arial, sans-serif"> call VecDestroy(vec_coords,ierr)</font></div><div><font face="arial, sans-serif"> call MatDestroy(Matnull,ierr)</font></div><div><font face="arial, sans-serif"> endif</font></div>
<div><font face="arial, sans-serif"> call KSPDestroy(ksp,ierr)</font></div><div><font face="arial, sans-serif"> call PetscTime(t0,ierr)</font></div><div><font face="arial, sans-serif"> write(*,*) "time celan up :", t0-t1</font></div>
<div><br></div><div><font face="arial, sans-serif"> end</font></div><div style="font-family:arial,sans-serif;font-size:13px"><br></div></div><div class="gmail_extra">-- <br>Einar Sørheim
</div></div></div>
</div></div><span><Mail Attachment></span></blockquote></div><br></div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br>Einar Sørheim
</div>
</blockquote></div><br></div></body></html>