<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">Cong:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hong, <div><br><div>Sure.</div><div><br></div><div><span style="font-size:13px">I want to extend the Krylov subspace by </span><span style="font-size:13px">step_k</span><span style="font-size:13px"> dimensions by using monomial, which can be defined as </span></div><div><span style="font-size:13px"><br></span></div><div><span style="font-size:13px">K={Km(1)m Km(2), ..., Km(</span><span style="font-size:13px">step_k</span><span style="font-size:13px">)}</span></div><div><span style="font-size:13px"> ={Km(1), AKm(1), AKm(2), ... , AKm(step_k-1)}</span></div><div><span style="font-size:13px"> ={R, AR, A^2R, ... A^(</span><span style="font-size:13px">step_k-1</span><span style="font-size:13px">)R}</span></div></div></div></blockquote><div><br></div><div>A subspace with dense matrices as basis?</div><div>How large <span style="font-size:13px">step_k and </span>your matrices will be? </div><div><br></div><div>Hong</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Aug 6, 2015 at 12:23 PM, Hong <span dir="ltr"><<a href="mailto:hzhang@mcs.anl.gov" target="_blank">hzhang@mcs.anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Cong,</div><div><br></div>Can you write out math equations for <span style="font-size:12.8000001907349px">mpk_monomial (), </span><div><span style="font-size:12.8000001907349px">list input and output parameters. </span></div><div><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px">Note: </span></div><div><span style="font-size:12.8000001907349px">1. </span><span style="font-size:12.8000001907349px">MatDuplicate() does not need to be followed by MatAssemblyBegin/End</span></div><div><span style="font-size:12.8000001907349px">2. </span><span style="font-size:12.8000001907349px">MatMatMult(A,Km(stepIdx-1),</span><span style="font-size:12.8000001907349px">MAT_REUSE_MATRIX,..) must be called after</span></div><div><span style="font-size:12.8000001907349px"> MatMatMult(A,Km(stepIdx-1),</span><span style="font-size:12.8000001907349px">MAT_INITIAL_MATRIX,..)</span><span><font color="#888888"><span style="font-size:12.8000001907349px"><br></span></font></span></div><span><font color="#888888"><div><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px">Hong</span></div><div><span style="font-size:12.8000001907349px"><br></span></div></font></span></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Aug 5, 2015 at 8:56 PM, Cong Li <span dir="ltr"><<a href="mailto:solvercorleone@gmail.com" target="_blank">solvercorleone@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">The entire source code files are attached.<div><br></div><div>Also I copy and paste the here in this email</div><div><br></div><div>thanks</div><div><br></div><div><div>program test</div><div><br></div><div> implicit none</div><div><br></div><div>#include <finclude/petscsys.h></div><div>#include <finclude/petscvec.h></div><div>#include <finclude/petscmat.h></div><div>#include <finclude/petscviewer.h></div><div><br></div><div><br></div><div> PetscViewer :: view</div><div> ! sparse matrix</div><div> Mat :: A</div><div> ! distributed dense matrix of size n x m</div><div> Mat :: B, X, R, QDlt, AQDlt</div><div> ! distributed dense matrix of size n x (m x k)</div><div> Mat :: Q, K, AQ_p, AQ</div><div> ! local dense matrix (every process keep the identical copies), (m x k) x (m x k)</div><div> Mat :: AConjPara, QtAQ, QtAQ_p, Dlt</div><div><br></div><div> PetscInt :: nDim, mDim, rhsNDim,rhsMDim,ierr, maxIter, iter, step_k,bsize</div><div> PetscInt :: ownRowS,ownRowE</div><div> PetscScalar, allocatable :: XInit(:,:)</div><div> PetscInt :: XInitI, XInitJ</div><div> PetscScalar :: v=1.0</div><div> PetscBool :: flg</div><div> PetscMPIInt :: size, rank</div><div><br></div><div> character(128) :: fin, rhsfin</div><div><br></div><div><br></div><div> call PetscInitialize(PETSC_NULL_CHARACTER,ierr)</div><div> call MPI_Comm_size(PETSC_COMM_WORLD,size,ierr)</div><div> call MPI_Comm_rank(PETSC_COMM_WORLD,rank,ierr)</div><div><br></div><div> ! read binary matrix file</div><div> call PetscOptionsGetString(PETSC_NULL_CHARACTER,'-f',fin,flg,ierr)</div><div> call PetscOptionsGetString(PETSC_NULL_CHARACTER,'-r',rhsfin,flg,ierr)</div><div><br></div><div> call PetscOptionsGetInt(PETSC_NULL_CHARACTER,'-i',maxIter,flg,ierr)</div><div> call PetscOptionsGetInt(PETSC_NULL_CHARACTER,'-k',step_k,flg,ierr)</div><div> call PetscOptionsGetInt(PETSC_NULL_CHARACTER,'-w',bsize,flg,ierr)</div><div><br></div><div><br></div><div> call PetscViewerBinaryOpen(PETSC_COMM_WORLD,fin,FILE_MODE_READ,view,ierr)</div><div> call MatCreate(PETSC_COMM_WORLD,A,ierr)</div><div> call MatSetType(A,MATAIJ,ierr)</div><div> call MatLoad(A,view,ierr)</div><div> call PetscViewerDestroy(view,ierr)</div><div> ! for the time being, assume mDim == nDim is true</div><div> call MatGetSize(A, nDim, mDim, ierr)</div><div><br></div><div> if (rank == 0) then</div><div> print*,'Mat Size = ', nDim, mDim</div><div> end if</div><div><br></div><div> call MatAssemblyBegin(A, MAT_FINAL_ASSEMBLY, ierr)</div><div> call MatAssemblyEnd(A, MAT_FINAL_ASSEMBLY, ierr)</div><div> call MatGetOwnershipRange(A,ownRowS,ownRowE, ierr)</div><div><br></div><div> ! create right-and-side matrix</div><div> ! for the time being, choose row-wise decomposition</div><div> ! for the time being, assume nDim%size = 0</div><div> call MatCreateDense(PETSC_COMM_WORLD, (ownRowE - ownRowS), &</div><div> bsize, nDim, bsize,PETSC_NULL_SCALAR, B, ierr)</div><div> call PetscViewerBinaryOpen(PETSC_COMM_WORLD,rhsfin,FILE_MODE_READ,view, ierr)</div><div> call MatLoad(B,view,ierr)</div><div> call PetscViewerDestroy(view,ierr)</div><div> call MatGetSize(B, rhsMDim, rhsNDim, ierr)</div><div> if (rank == 0) then</div><div> print*,'MRHS Size actually are:', rhsMDim, rhsNDim</div><div> print*,'MRHS Size should be:', nDim, bsize</div><div> end if</div><div> call MatAssemblyBegin(B, MAT_FINAL_ASSEMBLY, ierr)</div><div> call MatAssemblyEnd(B, MAT_FINAL_ASSEMBLY, ierr)</div><div><br></div><div> ! inintial value guses X</div><div> allocate(XInit(nDim,bsize))</div><div> do XInitI=1, nDim</div><div> do XInitJ=1, bsize</div><div> XInit(XInitI,XInitJ) = 1.0</div><div> end do</div><div> end do</div><div><br></div><div> call MatCreateDense(PETSC_COMM_WORLD, (ownRowE - ownRowS), &</div><div> bsize, nDim, bsize,XInit, X, ierr)</div><div><br></div><div> call MatAssemblyBegin(X, MAT_FINAL_ASSEMBLY, ierr)</div><div> call MatAssemblyEnd (X, MAT_FINAL_ASSEMBLY, ierr)</div><div><br></div><div><br></div><div> ! B, X, R, QDlt, AQDlt</div><div> call MatDuplicate(B, MAT_DO_NOT_COPY_VALUES, R, ierr)</div><div> call MatAssemblyBegin(R, MAT_FINAL_ASSEMBLY, ierr)</div><div> call MatAssemblyEnd (R, MAT_FINAL_ASSEMBLY, ierr)</div><div><br></div><div> call MatDuplicate(B, MAT_DO_NOT_COPY_VALUES, QDlt, ierr)</div><div> call MatAssemblyBegin(QDlt, MAT_FINAL_ASSEMBLY, ierr)</div><div> call MatAssemblyEnd (QDlt, MAT_FINAL_ASSEMBLY, ierr)</div><div><br></div><div> call MatDuplicate(B, MAT_DO_NOT_COPY_VALUES, AQDlt, ierr)</div><div> call MatAssemblyBegin(AQDlt, MAT_FINAL_ASSEMBLY, ierr)</div><div> call MatAssemblyEnd (AQDlt, MAT_FINAL_ASSEMBLY, ierr)</div><div><br></div><div>! Q, K, AQ_p, AQ of size n x (m x k)</div><div> call MatCreateDense(PETSC_COMM_WORLD, (ownRowE - ownRowS), &</div><div> (bsize*step_k), nDim, (bsize*step_k),PETSC_NULL_SCALAR, Q, ierr)</div><div> call MatAssemblyBegin(Q, MAT_FINAL_ASSEMBLY, ierr)</div><div> call MatAssemblyEnd(Q, MAT_FINAL_ASSEMBLY, ierr)</div><div> </div><div> call MatDuplicate(Q, MAT_DO_NOT_COPY_VALUES, K, ierr)</div><div> call MatAssemblyBegin(K, MAT_FINAL_ASSEMBLY, ierr)</div><div> call MatAssemblyEnd(K, MAT_FINAL_ASSEMBLY, ierr)</div><div><br></div><div> call MatDuplicate(Q, MAT_DO_NOT_COPY_VALUES, AQ_p, ierr)</div><div> call MatAssemblyBegin(AQ_p, MAT_FINAL_ASSEMBLY, ierr)</div><div> call MatAssemblyEnd(AQ_p, MAT_FINAL_ASSEMBLY, ierr)</div><div><br></div><div> call MatDuplicate(Q, MAT_DO_NOT_COPY_VALUES, AQ, ierr)</div><div> call MatAssemblyBegin(AQ, MAT_FINAL_ASSEMBLY, ierr)</div><div> call MatAssemblyEnd(AQ, MAT_FINAL_ASSEMBLY, ierr)</div><div><br></div><div>! QtAQ, QtAQ_p, Dlt of size (m x k) x (m x k)</div><div> call MatCreateSeqDense(PETSC_COMM_SELF,(bsize*step_k),(bsize*step_k),&</div><div> PETSC_NULL_SCALAR, QtAQ, ierr)</div><div> call MatAssemblyBegin(QtAQ, MAT_FINAL_ASSEMBLY, ierr)</div><div> call MatAssemblyEnd (QtAQ, MAT_FINAL_ASSEMBLY, ierr)</div><div><br></div><div> call MatDuplicate(QtAQ, MAT_DO_NOT_COPY_VALUES, QtAQ_p , ierr)</div><div> call MatAssemblyBegin(QtAQ_p, MAT_FINAL_ASSEMBLY, ierr)</div><div> call MatAssemblyEnd (QtAQ_p, MAT_FINAL_ASSEMBLY, ierr)</div><div><br></div><div> call MatDuplicate(QtAQ, MAT_DO_NOT_COPY_VALUES, Dlt , ierr)</div><div> call MatAssemblyBegin(Dlt, MAT_FINAL_ASSEMBLY, ierr)</div><div> call MatAssemblyEnd (Dlt, MAT_FINAL_ASSEMBLY, ierr)</div><div><br></div><div> call MatDuplicate(QtAQ, MAT_DO_NOT_COPY_VALUES, AConjPara , ierr)</div><div> call MatAssemblyBegin(AConjPara, MAT_FINAL_ASSEMBLY, ierr)</div><div> call MatAssemblyEnd (AConjPara, MAT_FINAL_ASSEMBLY, ierr)</div><div><br></div><div>! calculation for R</div><div><br></div><div>! call matrix powers kernel</div><div> call mpk_monomial (K, A, R, step_k, rank,size)</div><div><br></div><div>! destory matrices</div><div> deallocate(XInit)</div><div><br></div><div> call MatDestroy(B, ierr)</div><div> call MatDestroy(X, ierr)</div><div> call MatDestroy(R, ierr)</div><div> call MatDestroy(QDlt, ierr)</div><div> call MatDestroy(AQDlt, ierr)</div><div> call MatDestroy(Q, ierr)</div><div> call MatDestroy(K, ierr)</div><div> call MatDestroy(AQ_p, ierr)</div><div> call MatDestroy(AQ, ierr)</div><div> call MatDestroy(QtAQ, ierr)</div><div> call MatDestroy(QtAQ_p, ierr)</div><div> call MatDestroy(Dlt, ierr)</div><div><br></div><div><br></div><div> call PetscFinalize(ierr)</div><div><br></div><div> stop</div><div><br></div><div>end program test</div></div><div><br></div><div><div><br></div><div>subroutine mpk_monomial (K, A, R, step_k, rank, sizeMPI)</div><div><span style="white-space:pre-wrap"> </span>implicit none</div><div><br></div><div>#include <finclude/petscsys.h></div><div>#include <finclude/petscvec.h></div><div>#include <finclude/petscmat.h></div><div>#include <finclude/petscviewer.h></div><div><br></div><div><span style="white-space:pre-wrap"> </span>Mat :: K, Km(step_k)</div><div><span style="white-space:pre-wrap"> </span>Mat :: A, R</div><div><span style="white-space:pre-wrap"> </span>PetscMPIInt :: sizeMPI, rank</div><div><span style="white-space:pre-wrap"> </span>PetscInt :: nDim, bsize, step_k, local_RRow, local_RCol, genIdx</div><div><span style="white-space:pre-wrap"> </span>PetscInt :: ierr</div><div><span style="white-space:pre-wrap"> </span>PetscInt :: stepIdx, blockShift, localRsize</div><div> PetscScalar :: KArray(1), RArray(1), PetscScalarSize</div><div> PetscOffset :: KArrayOffset, RArrayOffset</div><div><br></div><div><span style="white-space:pre-wrap"> </span>call MatGetSize(R, nDim, bsize, ierr)</div><div> <span style="white-space:pre-wrap"> </span>if (rank == 0) then</div><div> <span style="white-space:pre-wrap"> </span> print*,'Mat Size = ', nDim, bsize</div><div> <span style="white-space:pre-wrap"> </span>end if</div><div><br></div><div> call MatGetArray(K,KArray,KArrayOffset,ierr)</div><div><br></div><div> call MatGetLocalSize(R,local_RRow,local_RCol)</div><div>! print *, "local_RRow,local_RCol", local_RRow,local_RCol</div><div><br></div><div> ! get arry from R to add values to K(1)</div><span><div> call MatGetArray(R,RArray,RArrayOffset,ierr)</div><div><br></div><div> call MatCreateDense(PETSC_COMM_WORLD, PETSC_DECIDE, &</div><div> PETSC_DECIDE , nDim, bsize,KArray(KArrayOffset + 1), Km(1), ierr)</div><div><br></div><div><br></div></span><div>! call PetscMemmove(KArray(KArrayOffset + 1),RArray(RArrayOffset + 1) &</div><div>! ,local_RRow * local_RCol * STORAGE_SIZE(PetscScalarSize), ierr)</div><span><div><br></div><div> localRsize = local_RRow * local_RCol</div><div> do genIdx= 1, localRsize</div><div> KArray(KArrayOffset + genIdx) = RArray(RArrayOffset + genIdx)</div><div> end do</div><div><br></div><div><br></div><div> call MatRestoreArray(R,RArray,RArrayOffset,ierr)</div><div><br></div><div> call MatAssemblyBegin(Km(1), MAT_FINAL_ASSEMBLY, ierr)</div><div> call MatAssemblyEnd (Km(1), MAT_FINAL_ASSEMBLY, ierr)</div><div> </div><div> do stepIdx= 2, step_k</div><div><br></div><div> blockShift = KArrayOffset + (stepIdx-1) * (local_RRow * local_RCol)</div><div><br></div><div> call MatCreateDense(PETSC_COMM_WORLD, PETSC_DECIDE, &</div><div> PETSC_DECIDE , nDim, bsize,KArray(blockShift+1), Km(stepIdx), ierr)</div><div> call MatAssemblyBegin(Km(stepIdx), MAT_FINAL_ASSEMBLY, ierr)</div><div> call MatAssemblyEnd (Km(stepIdx), MAT_FINAL_ASSEMBLY, ierr)</div><div> </div><div> end do</div><div><br></div><div> call MatRestoreArray(K,KArray,KArrayOffset,ierr)</div><div><br></div></span><div>! do stepIdx= 2, step_k</div><div> do stepIdx= 2,2 </div><div><br></div><div> call MatMatMult(A,Km(stepIdx-1),MAT_REUSE_MATRIX,PETSC_DEFAULT_INTEGER,Km(stepIdx), ierr)</div><div>! call MatMatMult(A,Km(stepIdx-1),MAT_INITIAL_MATRIX,PETSC_DEFAULT_INTEGER,Km(stepIdx), ierr)</div><div> end do</div><div><br></div><div>! call MatView(K,PETSC_VIEWER_STDOUT_WORLD,ierr)</div><div><br></div><div>end subroutine mpk_monomial</div></div><span><font color="#888888"><div><br></div><div><br></div><div><br></div><div>Cong Li</div></font></span></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Aug 6, 2015 at 3:30 AM, Barry Smith <span dir="ltr"><<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
Send the entire code so that we can compile it and run it ourselves to see what is going wrong.<br>
<span><font color="#888888"><br>
Barry<br>
</font></span><div><div><br>
> On Aug 5, 2015, at 4:42 AM, Cong Li <<a href="mailto:solvercorleone@gmail.com" target="_blank">solvercorleone@gmail.com</a>> wrote:<br>
><br>
> Hi<br>
><br>
> I tried the method you suggested. However, I got the error message.<br>
> My code and message are below.<br>
><br>
> K is the big matrix containing column matrices.<br>
><br>
> code:<br>
><br>
> call MatGetArray(K,KArray,KArrayOffset,ierr)<br>
><br>
> call MatGetLocalSize(R,local_RRow,local_RCol)<br>
><br>
> call MatGetArray(R,RArray,RArrayOffset,ierr)<br>
><br>
> call MatCreateDense(PETSC_COMM_WORLD, PETSC_DECIDE, &<br>
> PETSC_DECIDE , nDim, bsize,KArray(KArrayOffset + 1), Km(1), ierr)<br>
><br>
> localRsize = local_RRow * local_RCol<br>
> do genIdx= 1, localRsize<br>
> KArray(KArrayOffset + genIdx) = RArray(RArrayOffset + genIdx)<br>
> end do<br>
><br>
> call MatRestoreArray(R,RArray,RArrayOffset,ierr)<br>
><br>
> call MatAssemblyBegin(Km(1), MAT_FINAL_ASSEMBLY, ierr)<br>
> call MatAssemblyEnd (Km(1), MAT_FINAL_ASSEMBLY, ierr)<br>
><br>
> do stepIdx= 2, step_k<br>
><br>
> blockShift = KArrayOffset + (stepIdx-1) * (local_RRow * local_RCol)<br>
><br>
> call MatCreateDense(PETSC_COMM_WORLD, PETSC_DECIDE, &<br>
> PETSC_DECIDE , nDim, bsize,KArray(blockShift+1), Km(stepIdx), ierr)<br>
> call MatAssemblyBegin(Km(stepIdx), MAT_FINAL_ASSEMBLY, ierr)<br>
> call MatAssemblyEnd (Km(stepIdx), MAT_FINAL_ASSEMBLY, ierr)<br>
> end do<br>
><br>
> call MatRestoreArray(K,KArray,KArrayOffset,ierr)<br>
><br>
> do stepIdx= 2, step_k<br>
><br>
> call MatMatMult(A,Km(stepIdx-1),MAT_REUSE_MATRIX,PETSC_DEFAULT_INTEGER,Km(stepIdx), ierr)<br>
> end do<br>
><br>
><br>
> And I got the error message as below:<br>
><br>
><br>
> [0]PETSC ERROR: ------------------------------------------------------------------------<br>
> [0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range<br>
> [0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger<br>
> [0]PETSC ERROR: or see <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[0]PETSC" rel="noreferrer" target="_blank">http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[0]PETSC</a> ERROR: or try <a href="http://valgrind.org" rel="noreferrer" target="_blank">http://valgrind.org</a> on GNU/linux and Apple Mac OS X to find memory corruption errors<br>
> [0]PETSC ERROR: configure using --with-debugging=yes, recompile, link, and run<br>
> [0]PETSC ERROR: to get more information on the crash.<br>
> [0]PETSC ERROR: --------------------- Error Message ------------------------------------<br>
> [0]PETSC ERROR: Signal received!<br>
> [0]PETSC ERROR: ------------------------------------------------------------------------<br>
> [0]PETSC ERROR: Petsc Release Version 3.3.0, Patch 7, Sat May 11 22:15:24 CDT 2013<br>
> [0]PETSC ERROR: See docs/changes/index.html for recent updates.<br>
> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.<br>
> [0]PETSC ERROR: See docs/index.html for manual pages.<br>
> [0]PETSC ERROR: --------------------[1]PETSC ERROR: ------------------------------------------------------------------------<br>
> [1]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range<br>
> ----------------------------------------------------<br>
> [0]PETSC ERROR: ./kmath.bcbcg on a arch-fuji named p01-024 by a03293 Wed Aug 5 18:24:40 2015<br>
> [0]PETSC ERROR: Libraries linked from /volume1/home/ra000005/a03293/kmathlibbuild/petsc-3.3-p7/arch-fujitsu-sparc64fx-opt/lib<br>
> [0]PETSC ERROR: Configure run at Tue Jul 28 19:23:51 2015<br>
> [0]PETSC ERROR: Configure options --known-level1-dcache-size=32768 --known-level1-dcache-linesize=32 --known-level1-dcache-assoc=0 --known-memcmp-ok=1 --known-sizeof-char=1 --known-sizeof-void-p=8 --known-sizeof-short=2 --known-sizeof-int=4 --known-sizeof-long=8 --known-sizeof-long-long=8 --known-sizeof-float=4 --known-sizeof-double=8 --known-sizeof-size_t=8 --known-bits-per-byte=8 --known-sizeof-MPI_Comm=8 --known-sizeof-MPI_Fint=4 --known-mpi-long-double=1 --known-mpi-c-double-complex=1 --with-cc=mpifccpx --CFLAGS="-mt -Xg" --COPTFLAGS=-Kfast,openmp --with-cxx=mpiFCCpx --CXXFLAGS=-mt --CXXOPTFLAGS=-Kfast,openmp --with-fc=mpifrtpx --FFLAGS=-Kthreadsafe --FOPTFLAGS=-Kfast,openmp --with-blas-lapack-lib="-SCALAPACK -SSL2" --with-x=0 --with-c++-support --with-batch=1 --with-info=1 --with-debugging=0 --known-mpi-shared-libraries=0 --with-valgrind=0<br>
> [0]PETSC ERROR: ------------------------------------------------------------------------<br>
> [0]PETSC ERROR: User provided function() line 0 in unknown directory unknown file<br>
> --------------------------------------------------------------------------<br>
> [mpi::mpi-api::mpi-abort]<br>
> MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD<br>
> with errorcode 59.<br>
><br>
> NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.<br>
> You may or may not see output from other processes, depending on<br>
> exactly when Open MPI kills them.<br>
> --------------------------------------------------------------------------<br>
> [p01-024:26516] /opt/FJSVtclang/GM-1.2.0-18/lib64/libmpi.so.0(orte_errmgr_base_error_abort+0x84) [0xffffffff0091f684]<br>
> [p01-024:26516] /opt/FJSVtclang/GM-1.2.0-18/lib64/libmpi.so.0(ompi_mpi_abort+0x51c) [0xffffffff006c389c]<br>
> [p01-024:26516] /opt/FJSVtclang/GM-1.2.0-18/lib64/libmpi.so.0(MPI_Abort+0x6c) [0xffffffff006db3ac]<br>
> [p01-024:26516] /opt/FJSVtclang/GM-1.2.0-18/lib64/libtrtmet_c.so.1(MPI_Abort+0x2c) [0xffffffff00281bf0]<br>
> [p01-024:26516] ./kmath.bcbcg [0x1bf620]<br>
> [p01-024:26516] ./kmath.bcbcg [0x1bf20c]<br>
> [p01-024:26516] /lib64/libc.so.6(killpg+0x48) [0xffffffff02d52600]<br>
> [p01-024:26516] [(nil)]<br>
> [p01-024:26516] ./kmath.bcbcg [0x1a2054]<br>
> [p01-024:26516] ./kmath.bcbcg [0x1064f8]<br>
> [p01-024:26516] ./kmath.bcbcg(MAIN__+0x9dc) [0x105d1c]<br>
> [p01-024:26516] ./kmath.bcbcg(main+0xec) [0x8a329c]<br>
> [p01-024:26516] /lib64/libc.so.6(__libc_start_main+0x194) [0xffffffff02d3b81c]<br>
> [p01-024:26516] ./kmath.bcbcg [0x1051ec]<br>
> [0]PETSC ERROR: ------------------------------------------------------------------------<br>
> [0]PETSC ERROR: Caught signal number 15 Terminate: Somet process (or the batch system) has told this process to end<br>
> [0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger<br>
> [0]PETSC ERROR: or see <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[0]PETSC" rel="noreferrer" target="_blank">http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[0]PETSC</a> ERROR: or try <a href="http://valgrind.org" rel="noreferrer" target="_blank">http://valgrind.org</a> on GNU/linux and Apple Mac OS X to find memory corruption errors<br>
> [0]PETSC ERROR: configure using --with-debugging=yes, recompile, link, and run<br>
> [0]PETSC ERROR: to get more information on the crash.<br>
> [0]PETSC ERROR: --------------------- Error Message ------------------------------------<br>
> [0]PETSC ERROR: Signal received!<br>
> [0]PETSC ERROR: ------------------------------------------------------------------------<br>
> [0]PETSC ERROR: Petsc Release Version 3.3.0, Patch 7, Sat May 11 22:15:24 CDT 2013<br>
> [0]PETSC ERROR: See docs/changes/index.html for recent updates.<br>
> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.<br>
> [0]PETSC ERROR: See docs/index.html for manual pages.<br>
> [0]PETSC ERROR: ------------------------------------------------------------------------<br>
> [0]PETSC ERROR: ./kmath.bcbcg on a arch-fuji named p01-024 by a03293 Wed Aug 5 18:24:40 2015<br>
> [0]PETSC ERROR: Libraries linked from /volume1/home/ra000005/a03293/kmathlibbuild/petsc-3.3-p7/arch-fujitsu-sparc64fx-opt/lib<br>
> [0]PETSC ERROR: Configure run at Tue Jul 28 19:23:51 2015<br>
> [0]PETSC ERROR: Configure options --known-level1-dcache-size=32768 --known-level1-dcache-linesize=32 --known-level1-dcache-assoc=0 --known-memcmp-ok=1 --known-sizeof-char=1 --known-sizeof-void-p=8 --known-sizeof-short=2 --known-sizeof-int=4 --known-sizeof-long=8 --known-sizeof-long-long=8 --known-sizeof-float=4 --known-sizeof-double=8 --known-sizeof-size_t=8 --known-bits-per-byte=8 --known-sizeof-MPI_Comm=8 --known-sizeof-MPI_Fint=4 --known-mpi-long-double=1 --known-mpi-c-double-complex=1 --with-cc=mpifccpx --CFLAGS="-mt -Xg" --COPTFLAGS=-Kfast,openmp --with-cxx=mpiFCCpx --CXXFLAGS=-mt --CXXOPTFLAGS=-Kfast,openmp --with-fc=mpifrtpx --FFLAGS=-Kthreadsafe --FOPTFLAGS=-Kfast,openmp --with-blas-lapack-lib="-SCALAPACK -SSL2" --with-x=0 --with-c++-support --with-batch=1 --with-info=1 --with-debugging=0 --known-mpi-shared-libraries=0 --with-valgrind=0<br>
> [0]PETSC ERROR: ------------------------------------------------------------------------<br>
> [0]PETSC ERROR: User provided function() line 0 in unknown directory unknown file<br>
> [ERR.] PLE 0019 plexec One of MPI processes was aborted.(rank=0)(nid=0x020a0028)(CODE=1938,793745140674134016,15104)<br>
><br>
> However, if I change from<br>
> call MatMatMult(A,Km(stepIdx-1),MAT_REUSE_MATRIX,PETSC_DEFAULT_INTEGER,Km(stepIdx), ierr)<br>
> to<br>
> call MatMatMult(A,Km(stepIdx-1), MAT_INITIAL_MATRIX,PETSC_DEFAULT_INTEGER,Km(stepIdx), ierr)<br>
><br>
> everything is fine.<br>
><br>
> could you please suggest some way to solve this?<br>
><br>
> Thanks<br>
><br>
> Cong Li<br>
><br>
> On Wed, Aug 5, 2015 at 10:53 AM, Cong Li <<a href="mailto:solvercorleone@gmail.com" target="_blank">solvercorleone@gmail.com</a>> wrote:<br>
> Thank you very much for your help and suggestions.<br>
> With your help, finally I could continue my project.<br>
><br>
> Regards<br>
><br>
> Cong Li<br>
><br>
><br>
><br>
> On Wed, Aug 5, 2015 at 3:09 AM, Barry Smith <<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>> wrote:<br>
><br>
> From the manual page: Unless scall is MAT_REUSE_MATRIX C will be created.<br>
><br>
> Since you want to use the C that is passed in you should use MAT_REUSE_MATRIX.<br>
><br>
> Note that since your B and C matrices are dense the issue of sparsity pattern of C is not relevant.<br>
><br>
> Barry<br>
><br>
> > On Aug 4, 2015, at 11:59 AM, Cong Li <<a href="mailto:solvercorleone@gmail.com" target="_blank">solvercorleone@gmail.com</a>> wrote:<br>
> ><br>
> > Thanks very much. This answer is very helpful.<br>
> > And I have a following question.<br>
> > If I create B1, B2, .. by the way you suggested and then use MatMatMult to do SPMM.<br>
> > PetscErrorCode MatMatMult(Mat A,Mat B,MatReuse scall,PetscReal fill,Mat *C)<br>
> > should I use MAT_REUSE_MATRIX for MatReuse part of the arguement.<br>
> ><br>
> > Thanks<br>
> ><br>
> > Cong Li<br>
> ><br>
> > On Wed, Aug 5, 2015 at 1:27 AM, Barry Smith <<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>> wrote:<br>
> ><br>
> > > On Aug 4, 2015, at 4:09 AM, Cong Li <<a href="mailto:solvercorleone@gmail.com" target="_blank">solvercorleone@gmail.com</a>> wrote:<br>
> > ><br>
> > > I am sorry that I should have explained it more clearly.<br>
> > > Actually I want to compute a recurrence.<br>
> > ><br>
> > > Like, I want to firstly compute A*X1=B1, and then calculate A*B1=B2, A*B2=B3 and so on.<br>
> > > Finally I want to combine all these results into a bigger matrix C=[B1,B2 ...]<br>
> ><br>
> > First create C with MatCreateDense(,&C). Then call MatDenseGetArray(C,&array); then create B1 with MatCreateDense(....,array,&B1); then create<br>
> > B2 with MatCreateDense(...,array+shift,&B2) etc where shift equals the number of __local__ rows in B1 times the number of columns in B1, then create B3 with a larger shift etc.<br>
> ><br>
> > Note that you are "sharing" the array space of C with B1, B2, B3, ..., each Bi contains its columns of the C matrix.<br>
> ><br>
> > Barry<br>
> ><br>
> ><br>
> ><br>
> > ><br>
> > > Is there any way to do this efficiently.<br>
> > ><br>
> > ><br>
> > ><br>
> > > On Tue, Aug 4, 2015 at 5:45 PM, Patrick Sanan <<a href="mailto:patrick.sanan@gmail.com" target="_blank">patrick.sanan@gmail.com</a>> wrote:<br>
> > > On Tue, Aug 04, 2015 at 03:42:14PM +0900, Cong Li wrote:<br>
> > > > Thanks for your reply.<br>
> > > ><br>
> > > > I have an other question.<br>
> > > > I want to do SPMM several times and combine result matrices into one bigger<br>
> > > > matrix.<br>
> > > > for example<br>
> > > > I firstly calculate AX1=B1, AX2=B2 ...<br>
> > > > then I want to combine B1, B2.. to get a C, where C=[B1,B2...]<br>
> > > ><br>
> > > > Could you please suggest a way of how to do this.<br>
> > > This is just linear algebra, nothing to do with PETSc specifically.<br>
> > > A * [X1, X2, ... ] = [AX1, AX2, ...]<br>
> > > ><br>
> > > > Thanks<br>
> > > ><br>
> > > > Cong Li<br>
> > > ><br>
> > > > On Tue, Aug 4, 2015 at 3:27 PM, Jed Brown <<a href="mailto:jed@jedbrown.org" target="_blank">jed@jedbrown.org</a>> wrote:<br>
> > > ><br>
> > > > > Cong Li <<a href="mailto:solvercorleone@gmail.com" target="_blank">solvercorleone@gmail.com</a>> writes:<br>
> > > > ><br>
> > > > > > Hello,<br>
> > > > > ><br>
> > > > > > I am a PhD student using PETsc for my research.<br>
> > > > > > I am wondering if there is a way to implement SPMM (Sparse matrix-matrix<br>
> > > > > > multiplication) by using PETSc.<br>
> > > > ><br>
> > > > ><br>
> > > > > <a href="http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatMatMult.html" rel="noreferrer" target="_blank">http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatMatMult.html</a><br>
> > > > ><br>
> > ><br>
> ><br>
> ><br>
><br>
><br>
><br>
<br>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div></div>