<div dir="ltr">Hi matt,<div><br></div><div>I'm not generating values on random processors.</div><div>here is the relevant part of the code:</div><div><br></div><div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><br> ierr = MatGetOwnershipRange(A,&Istart,&Iend);CHKERRQ(ierr);<br> MPI_Comm_rank(MPI_COMM_WORLD,&mpiid);<br> for (i=Istart; i<Iend; i++) {<br> tcountcol=0;<br> iii=i+1;<br> unit_l1_(<br> l1,<br> l2, <br> ktyp,<br> &iii,<br> xjjxy,<br> xjjz ,<br> xtt ,<br> &tcountcol,<br> &ntrou,<br> tcol,<br> val);<br> for(kk=0;kk<52;kk++){<br> value[kk] = val[kk];<br> col[kk] = tcol[kk]-1;<br> } <br> countcol=tcountcol;<br> ierr = MatSetValues(A,1,&i,countcol,col,value,INSERT_VALUES);CHKERRQ(ierr); <br> }</blockquote></blockquote></div><div><div><br></div><div>As can be seen, I get the relevant rows attributed to each processor and only MatSetValues() for the rows owned by the corresponding processors.</div></div><div>That's why it is surprising that MatAssemnly takes a long time !</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, May 8, 2015 at 12:21 PM, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class="">On Fri, May 8, 2015 at 3:27 AM, Vijay Gopal Chilkuri <span dir="ltr"><<a href="mailto:vijay.gopal.c@gmail.com" target="_blank">vijay.gopal.c@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Thanks for testing.<div><br></div><div>My problem is not with MatSetValues(), it is MatAssemblyBegin()/MatAssemblyEnd() that takes forever to finish.</div><div>MatSetValues() is quite fast as you noticed.</div><div><br></div><div>Actually, unit_l1_ is the subroutine that generates the nonzero terms (per row) of my hamiltonian. For large matrices this should be time consuming if done on a single processor. </div><div>This is the reason that i use a lot of processors to divide the work of calling unit_l1_ multiple times.</div><div>For example, if I use 171 processors over 10 nodes, unit_l1_ finishes in about 40 min. It is after this step at the MatAssemblyBegin() and MatAssemblyEnd() that the </div><div>PETSc code get's stuck.</div></div></blockquote><div><br></div></span><div>Are you generating values on a random process, or on the process that owns the row? If you</div><div>generate them anywhere, then MatAssemblyEnd() can have a large amount of data to</div><div>communicate.</div><div><br></div><div> Matt</div><div><div class="h5"><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>If it's not too much trouble, could you perhaps test with more processors ?</div><div><br></div><div>thanks again.</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, May 7, 2015 at 9:25 PM, Barry Smith <span dir="ltr"><<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
I built the code with gfortran and optimization on my mac and run it on one process. It is spending all its time in the unit_l1_ routine and almost no time in the MatSetValues() in fact if I comment out the MatSetValues() line it runs just as slow as before. How can you know the time is spend in the MatSetValues()? Have you timed it also with the MatSetValues() time commented out?<br>
<br>
Barry<br>
<div><div><br>
> On May 2, 2015, at 5:58 PM, Vijay Gopal Chilkuri <<a href="mailto:vijay.gopal.c@gmail.com" target="_blank">vijay.gopal.c@gmail.com</a>> wrote:<br>
><br>
> You are right ! I am using the IRPF90 tool to write my program in FORTRAN.<br>
><br>
> To use IRPF90 you can clone the github repository and get the irpf90 binary from the bin directory.<br>
> The code to generate the library that contains UNIT_L1_ can be found at: <a href="https://github.com/v1j4y/slepc_version.git" target="_blank">https://github.com/v1j4y/slepc_version.git</a><br>
> just clone this repository and do:<br>
><br>
> make<br>
> make irpf90.a<br>
><br>
> That's it !<br>
><br>
> I'm resending you the PETSc files along with the makefile. you'll need the irpf90.a library to make the ex1 executable.<br>
><br>
> Hopefully this time it'll work.<br>
> Please let me know if something goes wrong.<br>
><br>
> On Sun, May 3, 2015 at 12:24 AM, Barry Smith <<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>> wrote:<br>
><br>
> Our stupid mail system blocked the attachment because it contained irpf90.a Is this a library generated with <a href="https://github.com/scemama/irpf90" target="_blank">https://github.com/scemama/irpf90</a> ?<br>
> You need to email everything that goes into making that library because it is unlikely your .a library will work on my machine. What is this thing and can you just tar up the directory that builds it and I can generate irpf90.a<br>
><br>
> Barry<br>
><br>
> > On May 2, 2015, at 3:49 PM, Vijay Gopal Chilkuri <<a href="mailto:vijay.gopal.c@gmail.com" target="_blank">vijay.gopal.c@gmail.com</a>> wrote:<br>
> ><br>
> > Ok so here goes.<br>
> ><br>
> > I've attached a tarball with a directory which contains the required files to compile ex1.c<br>
> > The UNIT_L1_ subroutine generates the nonzero elements of my hamiltonian given a row, it is contained in the<br>
> > irpf90.a library. The actual fortran code is quite ugly.<br>
> > Please tell me if you are able to compile and run the codes.<br>
> ><br>
> > Thanks a lot,<br>
> > Vijay<br>
> ><br>
> > On Sat, May 2, 2015 at 10:26 PM, Barry Smith <<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>> wrote:<br>
> ><br>
> > > On May 2, 2015, at 2:57 PM, Vijay Gopal Chilkuri <<a href="mailto:vijay.gopal.c@gmail.com" target="_blank">vijay.gopal.c@gmail.com</a>> wrote:<br>
> > ><br>
> > > Please let me clarify,<br>
> > ><br>
> > > @Matt:<br>
> > > I've followed the proper manner to rapid assembly.<br>
> > > That is :<br>
> > > 1. preallocation<br>
> > > 2. nonzero ownership<br>
> > > Despite of having done the proper preallocation (as you can see from the output file),<br>
> > > the matrix takes ages to assemble, whereas the same code works for a larger matrix with (100,000,000) elements taking 2 min to assemble !<br>
> > ><br>
> > > @Barry:<br>
> > > I am not touching any file in the code that I attachd.<br>
> > > What i'm doing is that each processor will (according to it's ownership range) get it's nonzero entries<br>
> > > using a Fortran subroutine (UNIT_L1_) and call MatSetValues() for their respective rows.<br>
> > > Again to be clear I"m NOT doing file I/O anywhere.<br>
> ><br>
> > Oh sorry, where is the fortran code? I assumed the fortran code was reading from a file.<br>
> ><br>
> > Based on what you sent yes it should take little time; if you send something I could run I could try to see why it would be slow.<br>
> ><br>
> > Barry<br>
> ><br>
> > ><br>
> > > Could you please have a look at the files attached.<br>
> > ><br>
> > ><br>
> > ><br>
> > ><br>
> > ><br>
> > > On Sat, May 2, 2015 at 9:42 PM, Barry Smith <<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>> wrote:<br>
> > ><br>
> > > You have two hundred processes banging on the same file on the same filesystem fighting with each other to read in the matrix. That's like hiring two hundred guys to dig a ditch but making them share one shovel; you won't get speed up, you'll get speed down.<br>
> > ><br>
> > ><br>
> > > Make a simple sequential PETSc program that reads in the matrix and then store it with MatView() in binary format. Then in your parallel program use MatLoad() to read the matrix in (it will take a few seconds to read in the matrix in that case).<br>
> > ><br>
> > > Barry<br>
> > ><br>
> > ><br>
> > > > On May 2, 2015, at 2:10 PM, Vijay Gopal Chilkuri <<a href="mailto:vijay.gopal.c@gmail.com" target="_blank">vijay.gopal.c@gmail.com</a>> wrote:<br>
> > > ><br>
> > > > Hi,<br>
> > > ><br>
> > > > I'm trying to diagonalize large matrices using PETSc and SLEPc.<br>
> > > > I've successfully diagonalized 100 million dimensional matrix in 2hours.<br>
> > > ><br>
> > > > But, the problem is when i try to diagonailze (similar) a smaller metrix of<br>
> > > > dimension 67,000,000 (24 nonzero elements per row) with 10 noes running 20 processors each (similar to the 100 million case), the matrix assembly itself<br>
> > > > takes 2h !<br>
> > > ><br>
> > > > Could someone point out the mistakes that i'm making ?<br>
> > > ><br>
> > > > i attach the source code and the output with the mail.<br>
> > > ><br>
> > > > thanks,<br>
> > > > Vijay<br>
> > > ><br>
> > > ><br>
> > > > #include <slepceps.h><br>
> > > > #include <petsctime.h><br>
> > > ><br>
> > > > #undef __FUNCT__<br>
> > > > #define __FUNCT__ "main"<br>
> > > > int main(int argc,char **argv)<br>
> > > > {<br>
> > > > Mat A; /* problem matrix */<br>
> > > > EPS eps; /* eigenproblem solver context */<br>
> > > > EPSType type;<br>
> > > > PetscReal error,tol,re,im;<br>
> > > > PetscScalar kr,ki,value[52];<br>
> > > > Vec xr,xi;<br>
> > > > PetscInt n=16224936,ii,i,veclen,j,Istart,Iend,col[52],nev,maxit,its,nconv,countcol;<br>
> > > > PetscInt d_nz,o_nz;<br>
> > > > PetscLogDouble t1,t2,tt1,tt2;<br>
> > > > PetscBool FirstBlock=PETSC_FALSE,LastBlock=PETSC_FALSE;<br>
> > > > PetscErrorCode ierr;<br>
> > > > PetscScalar eigr;<br>
> > > > PetscScalar eigi;<br>
> > > > PetscScalar * data;<br>
> > > > Vec Vr,Vi;<br>
> > > > char filename[PETSC_MAX_PATH_LEN]="FIL666";<br>
> > > > PetscViewer viewer;<br>
> > > > PetscBool ishermitian;<br>
> > > > int mpiid;<br>
> > > > long int kk,iii;<br>
> > > > long int tcountcol,tcol[52];<br>
> > > > float val[52];<br>
> > > > long int ntrou=1;<br>
> > > > long int l1[52]={1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 };<br>
> > > > long int l2[52]={2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 24, 23, 22, 21, 20, 19, 18, 17, 16, 15, 14, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 };<br>
> > > > long int ktyp[52]={1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 };<br>
> > > > double xjjz[52] ={0.0333333333333,-0.8, 0.,<br>
> > > > 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};<br>
> > > > double xjjxy[52] ={0.0333333333333,-0.8,0.,<br>
> > > > 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};<br>
> > > > double xtt[52] ={-1.0,0.,0.,<br>
> > > > 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};<br>
> > > ><br>
> > > > SlepcInitialize(&argc,&argv,(char*)0,NULL);<br>
> > > > ierr = PetscPrintf(PETSC_COMM_WORLD,"\n1-D t-J Eigenproblem, n=%D\n\n",n);CHKERRQ(ierr);<br>
> > > > ierr = MatCreate(PETSC_COMM_WORLD,&A);CHKERRQ(ierr);<br>
> > > > //ierr = MatSetSizes(A,PETSC_DECIDE,PETSC_DECIDE,n,n);CHKERRQ(ierr);<br>
> > > > ierr = MatCreateAIJ(PETSC_COMM_WORLD,PETSC_DECIDE,PETSC_DECIDE,n,n,46,NULL,46,NULL,&A);CHKERRQ(ierr);<br>
> > > > ierr = MatSetFromOptions(A);CHKERRQ(ierr);<br>
> > > > ierr = MatSetUp(A);CHKERRQ(ierr);<br>
> > > ><br>
> > > > ierr = MatGetOwnershipRange(A,&Istart,&Iend);CHKERRQ(ierr);<br>
> > > > ierr = MatCreateAIJ(PETSC_COMM_WORLD,Iend-Istart,Iend-Istart,n,n,46,NULL,46,NULL,&A);CHKERRQ(ierr);<br>
> > > > MPI_Comm_rank(MPI_COMM_WORLD,&mpiid);<br>
> > > > ierr = PetscTime(&tt1);CHKERRQ(ierr);<br>
> > > > for (i=Istart; i<Iend; i++) {<br>
> > > > tcountcol=0;<br>
> > > > iii=i+1;<br>
> > > > if(i%5 == 0 && mpiid==0){<br>
> > > > ierr = PetscTime(&t1);CHKERRQ(ierr);<br>
> > > > }<br>
> > > > unit_l1_(<br>
> > > > l1,<br>
> > > > l2,<br>
> > > > ktyp,<br>
> > > > &iii,<br>
> > > > xjjxy,<br>
> > > > xjjz ,<br>
> > > > xtt ,<br>
> > > > &tcountcol,<br>
> > > > &ntrou,<br>
> > > > tcol,<br>
> > > > val);<br>
> > > > for(kk=0;kk<52;kk++){<br>
> > > > value[kk] = val[kk];<br>
> > > > col[kk] = tcol[kk]-1;<br>
> > > > }<br>
> > > > if(i%5 == 0 && mpiid==0){<br>
> > > > ierr = PetscTime(&t2);CHKERRQ(ierr);<br>
> > > > ierr = PetscPrintf(PETSC_COMM_WORLD," i: %d\n mpiid: %d\ntime: %f\n",i,mpiid,t2-t1);CHKERRQ(ierr);<br>
> > > > }<br>
> > > > countcol=tcountcol;<br>
> > > > if(i%5 == 0 && mpiid==0){<br>
> > > > ierr = PetscTime(&t1);CHKERRQ(ierr);<br>
> > > > }<br>
> > > > ierr = MatSetValues(A,1,&i,countcol,col,value,INSERT_VALUES);CHKERRQ(ierr);<br>
> > > > if(i%5 == 0 && mpiid==0){<br>
> > > > ierr = PetscTime(&t2);CHKERRQ(ierr);<br>
> > > > ierr = PetscPrintf(PETSC_COMM_WORLD," processor \n mpiid: %d\ntime: %f\n",mpiid,t2-t1);CHKERRQ(ierr);<br>
> > > > }<br>
> > > > }<br>
> > > > ierr = PetscTime(&tt2);CHKERRQ(ierr);<br>
> > > > ierr = PetscPrintf(PETSC_COMM_WORLD," Time used to build the matrix: %f\n",tt2-tt1);CHKERRQ(ierr);<br>
> > > ><br>
> > > ><br>
> > > > ierr = PetscTime(&tt1);CHKERRQ(ierr);<br>
> > > > ierr = MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr);<br>
> > > > ierr = MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr);<br>
> > > > ierr = PetscTime(&tt2);CHKERRQ(ierr);<br>
> > > > ierr = PetscPrintf(PETSC_COMM_WORLD," Time used to assemble the matrix: %f\n",tt2-tt1);CHKERRQ(ierr);<br>
> > > > ierr = MatGetVecs(A,NULL,&xr);CHKERRQ(ierr);<br>
> > > > ierr = MatGetVecs(A,NULL,&xi);CHKERRQ(ierr);<br>
> > > ><br>
> > > > ierr = EPSCreate(PETSC_COMM_WORLD,&eps);CHKERRQ(ierr);<br>
> > > > ierr = EPSSetOperators(eps,A,NULL);CHKERRQ(ierr);<br>
> > > > ierr = EPSSetProblemType(eps,EPS_HEP);CHKERRQ(ierr);<br>
> > > > ierr = EPSSetWhichEigenpairs(eps,EPS_SMALLEST_REAL);CHKERRQ(ierr);<br>
> > > ><br>
> > > > ierr = EPSSetFromOptions(eps);CHKERRQ(ierr);<br>
> > > > tol = 1.e-8;<br>
> > > > maxit = 10000000;<br>
> > > > ierr = EPSSetTolerances(eps,tol,maxit);CHKERRQ(ierr);<br>
> > > ><br>
> > > > ierr = PetscTime(&t1);CHKERRQ(ierr);<br>
> > > > ierr = EPSSolve(eps);CHKERRQ(ierr);<br>
> > > > ierr = PetscTime(&t2);CHKERRQ(ierr);<br>
> > > > ierr = PetscPrintf(PETSC_COMM_WORLD," Time used: %f\n",t2-t1);CHKERRQ(ierr);<br>
> > > > ierr = EPSGetIterationNumber(eps,&its);CHKERRQ(ierr);<br>
> > > > ierr = PetscPrintf(PETSC_COMM_WORLD," Number of iterations of the method: %D\n",its);CHKERRQ(ierr);<br>
> > > > ierr = EPSGetType(eps,&type);CHKERRQ(ierr);<br>
> > > > ierr = PetscPrintf(PETSC_COMM_WORLD," Solution method: %s\n\n",type);CHKERRQ(ierr);<br>
> > > > ierr = EPSGetDimensions(eps,&nev,NULL,NULL);CHKERRQ(ierr);<br>
> > > > ierr = PetscPrintf(PETSC_COMM_WORLD," Number of requested eigenvalues: %D\n",nev);CHKERRQ(ierr);<br>
> > > > ierr = EPSGetTolerances(eps,&tol,&maxit);CHKERRQ(ierr);<br>
> > > > ierr = PetscPrintf(PETSC_COMM_WORLD," Stopping condition: tol=%.4g, maxit=%D\n",(double)tol,maxit);CHKERRQ(ierr);<br>
> > > ><br>
> > > > ierr = EPSGetConverged(eps,&nconv);CHKERRQ(ierr);<br>
> > > > ierr = EPSPrintSolution(eps,NULL);CHKERRQ(ierr);<br>
> > > > /*<br>
> > > > EPSGetConverged(eps,&nconv);<br>
> > > > if (nconv>0) {<br>
> > > > PetscViewerASCIIOpen(PETSC_COMM_WORLD,filename,&viewer);<br>
> > > > EPSIsHermitian(eps,&ishermitian);<br>
> > > > for (i=0;i<nconv;i++) {<br>
> > > > EPSGetEigenvector(eps,i,xr,xi);<br>
> > > > VecView(xr,viewer);<br>
> > > > #if !defined(PETSC_USE_COMPLEX)<br>
> > > > if (!ishermitian) { VecView(xi,viewer); }<br>
> > > > #endif<br>
> > > > }<br>
> > > > PetscViewerDestroy(&viewer);<br>
> > > > }<br>
> > > > */<br>
> > > > ierr = EPSDestroy(&eps);CHKERRQ(ierr);<br>
> > > > ierr = MatDestroy(&A);CHKERRQ(ierr);<br>
> > > > ierr = VecDestroy(&xr);CHKERRQ(ierr);<br>
> > > > ierr = VecDestroy(&xi);CHKERRQ(ierr);<br>
> > > > ierr = SlepcFinalize();<br>
> > > > return 0;<br>
> > > > }<br>
> > > ><br>
> > > > OUTPUT:<br>
> > > > Time used to build the matrix: 1914.729022<br>
> > > > [0] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [0] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [6] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [6] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [182] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [182] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [100] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [100] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [60] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [60] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [140] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [140] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [120] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [120] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [162] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [162] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [20] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [20] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [80] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [80] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [40] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [40] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [7] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [7] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [185] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [185] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [101] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [101] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [61] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [61] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [142] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [142] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [130] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [130] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [166] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [166] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [21] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [21] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [81] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [81] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [44] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [44] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [9] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [9] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [187] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [187] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [105] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [105] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [62] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [62] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [143] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [143] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [132] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [132] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [169] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [169] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [24] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [24] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [84] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [84] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [45] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [45] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [10] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [10] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [188] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [188] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [109] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [109] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [63] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [63] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [148] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [148] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [133] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [133] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [170] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [170] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [25] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [25] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [86] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [86] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [47] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [47] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [11] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [11] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [190] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [190] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [111] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [111] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [64] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [64] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [156] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [156] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [134] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [134] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [171] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [171] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [26] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [26] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [91] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [91] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [48] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [48] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [12] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [12] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [192] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [192] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [112] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [112] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [65] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [65] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [157] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [157] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [135] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [135] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [174] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [174] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [28] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [28] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [13] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [13] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [194] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [194] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [113] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [113] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [66] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [66] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [158] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [158] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [136] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [136] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [160] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [160] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [29] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [29] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [92] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [92] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [53] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [53] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [14] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [14] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [195] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [195] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [114] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [114] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [67] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [67] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [141] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [141] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [137] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [137] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [161] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [161] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [32] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [32] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [93] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [93] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [54] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [54] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [15] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [15] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [196] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [196] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [115] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [115] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [68] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [68] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [144] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [144] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [138] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [138] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [163] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [163] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [33] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [33] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [94] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [94] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [55] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [55] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [17] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [17] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [198] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [198] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [116] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [116] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [69] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [69] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [145] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [145] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [139] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [139] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [164] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [164] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [34] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [34] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [95] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [95] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [56] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [56] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [1] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [1] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [199] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [199] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [119] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [119] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [70] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [70] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [146] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [146] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [121] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [121] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [165] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [165] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [35] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [35] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [99] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [99] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [59] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [59] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [2] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [2] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [180] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [180] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [102] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [102] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [71] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [71] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [147] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [147] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [122] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [122] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [167] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [167] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [36] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [36] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [82] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [82] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [41] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [41] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [3] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [3] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [181] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [181] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [103] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [103] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [72] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [72] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [149] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [149] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [123] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [123] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [168] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [168] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [37] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [37] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [83] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [83] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [42] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [42] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [4] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [4] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [183] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [183] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [104] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [104] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [73] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [73] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [150] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [150] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [124] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [124] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [172] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [172] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [38] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [38] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [85] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [85] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [43] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [43] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [5] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [5] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [184] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [184] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [106] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [106] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [74] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [74] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [151] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [151] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [125] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [125] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [173] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [173] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [39] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [39] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [87] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [87] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [46] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [46] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [8] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [8] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [186] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [186] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [107] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [107] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [75] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [75] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [152] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [152] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [126] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [126] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [175] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [175] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [22] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [22] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [88] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [88] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [49] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [49] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [16] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [16] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [189] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [189] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [108] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [108] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [76] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [76] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [153] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [153] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [127] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [127] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [176] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [176] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [23] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [23] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [89] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [89] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [50] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [50] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [18] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [18] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [191] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [191] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [110] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [110] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [77] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [77] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [154] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [154] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [128] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [128] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [177] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [177] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [27] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [27] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [90] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [90] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [51] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [51] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [19] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [19] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [193] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [193] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [117] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [117] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [78] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [78] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [155] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [155] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [129] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [129] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [178] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [178] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [30] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [30] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [96] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [96] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [52] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [52] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [197] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [197] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [118] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [118] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [79] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [79] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [159] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [159] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [131] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [131] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [179] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [179] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [31] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [31] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [97] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [97] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [57] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [57] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [98] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [98] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [58] MatStashScatterBegin_Private(): No of messages: 0<br>
> > > > [58] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
> > > > [104] MatAssemblyEnd_SeqAIJ(): Matrix size: 81125 X 81125; storage space: 3238253 unneeded,493497 used<br>
> > > > [104] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0<br>
> > > > [104] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 12<br>
> > > > [104] MatCheckCompressedRow(): Found the ratio (num_zerorows 0)/(num_localrows 81125) < 0.6. Do not use CompressedRow routines.<br>
> > > > [4] MatAssemblyEnd_SeqAIJ(): Matrix size: 81125 X 81125; storage space: 3197999 unneeded,533751 used<br>
> > > > [4] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0<br>
> > > > [4] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 13<br>
> > > > [4] MatCheckCompressedRow(): Found the ratio (num_zerorows 0)/(num_localrows 81125) < 0.6. Do not use CompressedRow routines.<br>
> > > > [106] MatAssemblyEnd_SeqAIJ(): Matrix size: 81125 X 81125; storage space: 3243809 unneeded,487941 used<br>
> > > > [106] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0<br>
> > > > [106] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 12<br>
> > > > [106] MatCheckCompressedRow(): Found the ratio (num_zerorows 0)/(num_localrows 81125) < 0.6. Do not use CompressedRow routines.<br>
> > > > [144] MatAssemblyEnd_SeqAIJ(): Matrix size: 81124 X 81124; storage space: 3243446 unneeded,488258 used<br>
> > > > [144] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0<br>
> > > > [144] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 11<br>
> > > > [144] MatCheckCompressedRow(): Found the ratio (num_zerorows 0)/(num_localrows 81124) < 0.6. Do not use CompressedRow routines.<br>
> > > > [123] MatAssemblyEnd_SeqAIJ(): Matrix size: 81125 X 81125; storage space: 3237359 unneeded,494391 used<br>
> > > > [123] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0<br>
> > > > [123] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 12<br>
> > > > [123] MatCheckCompressedRow(): Found the ratio (num_zerorows 0)/(num_localrows 81125) < 0.6. Do not use CompressedRow routines.<br>
> > > > [173] MatAssemblyEnd_SeqAIJ(): Matrix size: 81124 X 81124; storage space: 3245082 unneeded,486622 used<br>
> > > > [173] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0<br>
> > > > [173] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 12<br>
> > > > [173] MatCheckCompressedRow(): Found the ratio (num_zerorows 0)/(num_localrows 81124) < 0.6. Do not use CompressedRow routines.<br>
> > > > [85] MatAssemblyEnd_SeqAIJ(): Matrix size: 81125 X 81125; storage space: 3217795 unneeded,513955 used<br>
> > > > [85] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0<br>
> > > > [85] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 12<br>
> > > > [85] MatCheckCompressedRow(): Found the ratio (num_zerorows 0)/(num_localrows 81125) < 0.6. Do not use CompressedRow routines.<br>
> > > > [6] MatAssemblyEnd_SeqAIJ(): Matrix size: 81125 X 81125; storage space: 3208633 unneeded,523117 used<br>
> > > > [6] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0<br>
> > > > [6] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 13<br>
> > > > [6] MatCheckCompressedRow(): Found the ratio (num_zerorows 0)/(num_localrows 81125) < 0.6. Do not use CompressedRow routines.<br>
> > > > [185] MatAssemblyEnd_SeqAIJ(): Matrix size: 81124 X 81124; storage space: 3209020 unneeded,522684 used<br>
> > > > [185] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0<br>
> > > > [185] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 12<br>
> > > > [185] MatCheckCompressedRow(): Found the ratio (num_zerorows 0)/(num_localrows 81124) < 0.6. Do not use CompressedRow routines.<br>
> > > > [108] MatAssemblyEnd_SeqAIJ(): Matrix size: 81125 X 81125; storage space: 3216305 unneeded,515445 used<br>
> > > > [108] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0<br>
> > > > [108] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 12<br>
> > > > [73] MatAssemblyEnd_SeqAIJ(): Matrix size: 81125 X 81125; storage space: 3201643 unneeded,530107 used<br>
> > > > [73] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0<br>
> > > > [73] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 13<br>
> > > > [73] MatCheckCompressedRow(): Found the ratio (num_zerorows 0)/(num_localrows 81125) < 0.6. Do not use CompressedRow routines.<br>
> > > > [146] MatAssemblyEnd_SeqAIJ(): Matrix size: 81124 X 81124; storage space: 3234040 unneeded,497664 used<br>
> > > > [146] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0<br>
> > > > [146] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 11<br>
> > > > [146] MatCheckCompressedRow(): Found the ratio (num_zerorows 0)/(num_localrows 81124) < 0.6. Do not use CompressedRow routines.<br>
> > > > [127] MatAssemblyEnd_SeqAIJ(): Matrix size: 81125 X 81125; storage space: 3243435 unneeded,488315 used<br>
> > > > [127] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0<br>
> > > > [127] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 12<br>
> > > > [127] MatCheckCompressedRow(): Found the ratio (num_zerorows 0)/(num_localrows 81125) < 0.6. Do not use CompressedRow routines.<br>
> > > > [177] MatAssemblyEnd_SeqAIJ(): Matrix size: 81124 X 81124; storage space: 3249858 unneeded,481846 used<br>
> > > > [177] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0<br>
> > > > [177] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 12<br>
> > > > [177] MatCheckCompressedRow(): Found the ratio (num_zerorows 0)/(num_localrows 81124) < 0.6. Do not use CompressedRow routines.<br>
> > > > [27] MatAssemblyEnd_SeqAIJ(): Matrix size: 81125 X 81125; storage space: 3207727 unneeded,524023 used<br>
> > > > [27] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0<br>
> > > > [27] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 13<br>
> > > > [27] MatCheckCompressedRow(): Found the ratio (num_zerorows 0)/(num_localrows 81125) < 0.6. Do not use CompressedRow routines.<br>
> > > > [87] MatAssemblyEnd_SeqAIJ(): Matrix size: 81125 X 81125; storage space: 3220647 unneeded,511103 used<br>
> > > > [87] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0<br>
> > > > [87] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 12<br>
> > > > [87] MatCheckCompressedRow(): Found the ratio (num_zerorows 0)/(num_localrows 81125) < 0.6. Do not use CompressedRow routines.<br>
> > > > [54] MatAssemblyEnd_SeqAIJ(): Matrix size: 81125 X 81125; storage space: 3197939 unneeded,533811 used<br>
> > > > [54] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0<br>
> > > > [54] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 13<br>
> > > > [54] MatCheckCompressedRow(): Found the ratio (num_zerorows 0)/(num_localrows 81125) < 0.6. Do not use CompressedRow routines.<br>
> > > > [8] MatAssemblyEnd_SeqAIJ(): Matrix size: 81125 X 81125; storage space: 3179441 unneeded,552309 used<br>
> > > > [8] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0<br>
> > > > [8] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 13<br>
> > > > [8] MatCheckCompressedRow(): Found the ratio (num_zerorows 0)/(num_localrows 81125) < 0.6. Do not use CompressedRow routines.<br>
> > > > [187] MatAssemblyEnd_SeqAIJ(): Matrix size: 81124 X 81124; storage space: 3197890 unneeded,533814 used<br>
> > > > [187] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0<br>
> > > > [187] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 13<br>
> > > > [187] MatCheckCompressedRow(): Found the ratio (num_zerorows 0)/(num_localrows 81124) < 0.6. Do not use CompressedRow routines.<br>
> > > > [108] MatCheckCompressedRow(): Found the ratio (num_zerorows 0)/(num_localrows 81125) < 0.6. Do not use CompressedRow routines.<br>
> > > > [74] MatAssemblyEnd_SeqAIJ(): Matrix size: 81125 X 81125; storage space: 3200381 unneeded,531369 used<br>
> > > ><br>
> > > > <ex1.c><inp.log><br>
> > ><br>
> > ><br>
> ><br>
> ><br>
> > <1_Warning.txt><br>
><br>
><br>
</div></div>> <ex1.c><makefile><br>
<br>
</blockquote></div><br></div>
</blockquote></div></div></div><br><br clear="all"><span class=""><div><br></div>-- <br><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</span></div></div>
</blockquote></div><br></div>