<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Hi all,<div><br></div><div>I am currently using PETSc to solve a system of linear equations in parallel, so I created a wrapper class that interfaces my C++ code with the PETSc library.</div><div><br></div><div>Since the problem I'm trying to solve is nonlinear, I assemble the matrix once and I reuse the structure (the sparsity pattern is the same). The problem is that after a few times of solving the system very fast, it suddenly takes a considerable amount of time and I really don't understand why this is happening.</div><div><br></div><div><div>This is an extract of the code showing the timing issue that I have just before solving the system with 2 processes:</div></div><div><br></div><div><br></div><div><div>[0] Inside PETSc solver:7.15256e-06 s</div><div>[0] Got ownership range... 3.21865e-05 s</div><div>Inside PETSc solver:1.5974e-05 s</div><div>[1] Got ownership range... 4.00543e-05 s</div><div>[1] Zeroed-out sparse matrix entries... [0] Zeroed-out sparse matrix entries... 0.000444174 s</div><div>0.000442982 s</div><div>[1] Filled sparse matrix... 0.0772262 s</div><div>[0] Filled sparse matrix... 0.081254 s</div><div>[1] Sparse matrix assembled... 0.0897892 s</div><div>[0] Sparse matrix assembled... 0.0954142 s</div><div>[0] Vectors created... 0.0955811 s[1] Vectors created... 0.095592 s</div><div><br></div><div>[0] Right hand side set... 0.0965121 s</div><div>[1] Right hand side set... 0.0965312 s</div><div>[0] Right-hand-side vector assembled... 0.096647 s</div><div>[1] Right-hand-side vector assembled... 0.096657 s</div><div>[0] System solved... 32.6188 s</div><div>[1] System solved... 32.6188 s</div><div>[0] Solution vector scattered... [1] Solution vector scattered... 32.6192 s</div><div>32.6192 s</div><div>[1] Solution vector copied... 32.6215[0] Solution vector copied... 32.6215 s</div><div> s</div><div>[1] Temporary data structures destroyed... 32.6216 s[0] Temporary data structures destroyed... 32.6216 s</div></div><div><br></div><div><br></div><div><br></div><div>So you can see that it takes a considerable amount of time to solve the system at this point, when usually this is what I get:</div><div><br></div><div>[0] Right-hand-side vector assembled... 0.089427 s</div><div>[1] Right-hand-side vector assembled... 0.089438 s</div><div>[0] System solved... 0.383509 s</div><div>[1] System solved... 0.38352 s</div><div><br></div><div><br></div><div>Does anyone know why this is happening? Thanks in advance for any insight on this.</div><div><br></div><div>Alejandro M. Aragón</div><div><br></div><div>P.S.: The code I use is given below:</div><div><br></div><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(77, 129, 134); "><span style="color: rgb(0, 0, 0); "><span class="Apple-tab-span" style="white-space: pre; "> </span></span><span style="color: rgb(140, 34, 95); ">struct</span><span style="color: rgb(0, 0, 0); "> </span>PETSc_solver<span style="color: rgb(0, 0, 0); "> {</span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "><span class="Apple-tab-span" style="white-space: pre; "> </span><br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span class="Apple-tab-span" style="white-space: pre; "> </span><span style="color: rgb(140, 34, 95); ">typedef</span> <span style="color: rgb(140, 34, 95); ">doVERBOSEuble</span> value_type;</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "><span class="Apple-tab-span" style="white-space: pre; "> </span><span style="color: rgb(140, 34, 95); ">typedef</span> sparse_vector<value_type> sparse_vector_type;</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "><span class="Apple-tab-span" style="white-space: pre; "> </span><span style="color: rgb(140, 34, 95); ">typedef</span> sparse_matrix<value_type> sparse_matrix_type;</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div></div><blockquote class="webkit-indent-blockquote" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 40px; border-top-style: none; border-right-style: none; border-bottom-style: none; border-left-style: none; border-width: initial; border-color: initial; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; "><blockquote class="webkit-indent-blockquote" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 40px; border-top-style: none; border-right-style: none; border-bottom-style: none; border-left-style: none; border-width: initial; border-color: initial; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; "><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(140, 34, 95); ">int</span> argc_;</div></div></blockquote><blockquote class="webkit-indent-blockquote" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 40px; border-top-style: none; border-right-style: none; border-bottom-style: none; border-left-style: none; border-width: initial; border-color: initial; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; "><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(140, 34, 95); ">char</span> **args_;</div></div></blockquote><blockquote class="webkit-indent-blockquote" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 40px; border-top-style: none; border-right-style: none; border-bottom-style: none; border-left-style: none; border-width: initial; border-color: initial; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; "><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> Mat A_; <span style="color: rgb(38, 126, 61); ">//!< linear system matrix</span></div></div></blockquote><blockquote class="webkit-indent-blockquote" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 40px; border-top-style: none; border-right-style: none; border-bottom-style: none; border-left-style: none; border-width: initial; border-color: initial; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; "><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> Vec x_; <span style="color: rgb(38, 126, 61); ">//!< Solution vector</span></div></div></blockquote><blockquote class="webkit-indent-blockquote" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 40px; border-top-style: none; border-right-style: none; border-bottom-style: none; border-left-style: none; border-width: initial; border-color: initial; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; "><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> KSP ksp_; <span style="color: rgb(38, 126, 61); ">//!< linear solver context</span></div></div></blockquote><blockquote class="webkit-indent-blockquote" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 40px; border-top-style: none; border-right-style: none; border-bottom-style: none; border-left-style: none; border-width: initial; border-color: initial; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; "><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(140, 34, 95); ">bool</span> allocated_;</div></div></blockquote></blockquote><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "><span class="Apple-tab-span" style="white-space: pre; "> </span>PETSc_solver() : argc_(), args_(<span style="color: rgb(140, 34, 95); ">NULL</span>), allocated_(<span style="color: rgb(140, 34, 95); ">false</span>) {</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div></div><blockquote class="webkit-indent-blockquote" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 40px; border-top-style: none; border-right-style: none; border-bottom-style: none; border-left-style: none; border-width: initial; border-color: initial; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; "><blockquote class="webkit-indent-blockquote" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 40px; border-top-style: none; border-right-style: none; border-bottom-style: none; border-left-style: none; border-width: initial; border-color: initial; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; "><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> PetscInitialize(&argc_,&args_,<span style="color: rgb(140, 34, 95); ">NULL</span>,<span style="color: rgb(140, 34, 95); ">NULL</span>);</div></div></blockquote><blockquote class="webkit-indent-blockquote" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 40px; border-top-style: none; border-right-style: none; border-bottom-style: none; border-left-style: none; border-width: initial; border-color: initial; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; "><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> PetscErrorCode <span style="color: rgb(77, 129, 134); ">ierr</span> = MatCreate(PETSC_COMM_WORLD,&A_);<span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>);</div></div></blockquote><blockquote class="webkit-indent-blockquote" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 40px; border-top-style: none; border-right-style: none; border-bottom-style: none; border-left-style: none; border-width: initial; border-color: initial; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; "><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(77, 129, 134); ">ierr</span> = KSPCreate(PETSC_COMM_WORLD,&ksp_);<span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>);</div></div></blockquote></blockquote><blockquote class="webkit-indent-blockquote" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 40px; border-top-style: none; border-right-style: none; border-bottom-style: none; border-left-style: none; border-width: initial; border-color: initial; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; "><blockquote class="webkit-indent-blockquote" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 40px; border-top-style: none; border-right-style: none; border-bottom-style: none; border-left-style: none; border-width: initial; border-color: initial; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; "><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> }</div></div></blockquote></blockquote><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "><span class="Apple-tab-span" style="white-space: pre; "> </span>PETSc_solver(<span style="color: rgb(140, 34, 95); ">int</span> argc, <span style="color: rgb(140, 34, 95); ">char</span>** args) : argc_(argc), args_(args) {</div></div><blockquote class="webkit-indent-blockquote" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 40px; border-top-style: none; border-right-style: none; border-bottom-style: none; border-left-style: none; border-width: initial; border-color: initial; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; "><blockquote class="webkit-indent-blockquote" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 40px; border-top-style: none; border-right-style: none; border-bottom-style: none; border-left-style: none; border-width: initial; border-color: initial; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; "><div><p style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></p></div></blockquote><blockquote class="webkit-indent-blockquote" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 40px; border-top-style: none; border-right-style: none; border-bottom-style: none; border-left-style: none; border-width: initial; border-color: initial; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; "><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> PetscInitialize(&argc_,&args_,<span style="color: rgb(140, 34, 95); ">NULL</span>,<span style="color: rgb(140, 34, 95); ">NULL</span>);</div></div></blockquote><blockquote class="webkit-indent-blockquote" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 40px; border-top-style: none; border-right-style: none; border-bottom-style: none; border-left-style: none; border-width: initial; border-color: initial; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; "><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> PetscErrorCode <span style="color: rgb(77, 129, 134); ">ierr</span> = MatCreate(PETSC_COMM_WORLD,&A_);<span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>);</div></div></blockquote><blockquote class="webkit-indent-blockquote" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 40px; border-top-style: none; border-right-style: none; border-bottom-style: none; border-left-style: none; border-width: initial; border-color: initial; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; "><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(77, 129, 134); ">ierr</span> = KSPCreate(PETSC_COMM_WORLD,&ksp_);<span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>);</div></div></blockquote><blockquote class="webkit-indent-blockquote" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 40px; border-top-style: none; border-right-style: none; border-bottom-style: none; border-left-style: none; border-width: initial; border-color: initial; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; "><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> }</div></div></blockquote></blockquote><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "><span style="color: rgb(0, 0, 0); "> </span>//! Overload operator() to solve system of linear equations</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(77, 129, 134); "><span style="color: rgb(0, 0, 0); "><span class="Apple-tab-span" style="white-space: pre; "> </span></span>sparse_vector_type<span style="color: rgb(0, 0, 0); "> </span><span style="color: rgb(140, 34, 95); ">operator</span><span style="color: rgb(0, 0, 0); ">()(</span><span style="color: rgb(140, 34, 95); ">const</span><span style="color: rgb(0, 0, 0); "> </span>sparse_matrix_type<span style="color: rgb(0, 0, 0); ">& </span>AA<span style="color: rgb(0, 0, 0); ">, </span><span style="color: rgb(140, 34, 95); ">const</span><span style="color: rgb(0, 0, 0); "> </span>sparse_vector_type<span style="color: rgb(0, 0, 0); ">& </span>bb<span style="color: rgb(0, 0, 0); ">);</span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> ~PETSc_solver() {</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> PetscErrorCode <span style="color: rgb(77, 129, 134); ">ierr</span> = MatDestroy(A_);<span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>);</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(77, 129, 134); ">ierr</span> = KSPDestroy(ksp_);<span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>);</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(77, 129, 134); ">ierr</span> = VecDestroy(x_);<span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>);</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> }</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> };</div></div><div><br></div><div><br></div><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); "><font class="Apple-style-span" color="#000000"><font class="Apple-style-span" color="#7D4831"><br></font></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(77, 129, 134); ">PETSc_solver<span style="color: rgb(0, 0, 0); ">::</span>sparse_vector_type<span style="color: rgb(0, 0, 0); "> </span>PETSc_solver<span style="color: rgb(0, 0, 0); ">::</span><span style="color: rgb(140, 34, 95); ">operator</span><span style="color: rgb(0, 0, 0); ">()(</span><span style="color: rgb(140, 34, 95); ">const</span><span style="color: rgb(0, 0, 0); "> </span>PETSc_solver<span style="color: rgb(0, 0, 0); ">::</span>sparse_matrix_type<span style="color: rgb(0, 0, 0); ">& </span>AA<span style="color: rgb(0, 0, 0); ">, </span><span style="color: rgb(140, 34, 95); ">const</span><span style="color: rgb(0, 0, 0); "> </span>PETSc_solver<span style="color: rgb(0, 0, 0); ">::</span>sparse_vector_type<span style="color: rgb(0, 0, 0); ">& </span>bb<span style="color: rgb(0, 0, 0); ">) {</span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> assert(<span style="color: rgb(77, 129, 134); ">AA</span>.<span style="color: rgb(48, 89, 93); ">rows</span>() == <span style="color: rgb(77, 129, 134); ">bb</span>.<span style="color: rgb(48, 89, 93); ">size</span>());</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> Vec <span style="color: rgb(77, 129, 134); ">x</span>,<span style="color: rgb(77, 129, 134); ">b</span>; <span style="color: rgb(38, 126, 61); ">/* approx solution, RHS */</span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "><span class="Apple-style-span" style="color: rgb(0, 0, 0); "> PetscInt <span style="color: rgb(77, 129, 134); ">Istart</span>,<span style="color: rgb(77, 129, 134); ">Iend</span>;</span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> PetscErrorCode <span style="color: rgb(77, 129, 134); ">ierr</span>;</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); ">#ifdef VERBOSE</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(77, 129, 134); "><span style="color: rgb(0, 0, 0); "> </span>cpputils<span style="color: rgb(0, 0, 0); ">::</span>ctimer<span style="color: rgb(0, 0, 0); "> </span>timer<span style="color: rgb(0, 0, 0); ">;</span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(158, 32, 32); "><span style="color: rgb(0, 0, 0); "> </span><span style="color: rgb(121, 72, 48); ">MPIPRINT</span><span style="color: rgb(0, 0, 0); ">(</span>"Inside PETSc solver:"<span style="color: rgb(0, 0, 0); "> );</span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> cout<<<span style="color: rgb(77, 129, 134); ">timer</span><<endl;</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); ">#endif</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <span style="color: rgb(77, 129, 134); ">ierr</span> = MatSetFromOptions(A_);<span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>);</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(140, 34, 95); ">if</span> (!allocated_) {</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); ">#ifdef VERBOSE</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(158, 32, 32); "><span style="color: rgb(0, 0, 0); "> </span><span style="color: rgb(121, 72, 48); ">MPIPRINT</span><span style="color: rgb(0, 0, 0); ">(</span>" Allocating memory... "<span style="color: rgb(0, 0, 0); ">);</span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> cout<<<span style="color: rgb(77, 129, 134); ">timer</span><<endl;</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); ">#endif</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(77, 129, 134); ">ierr</span> = MatSetSizes(A_,PETSC_DECIDE,PETSC_DECIDE, <span style="color: rgb(77, 129, 134); ">AA</span>.<span style="color: rgb(48, 89, 93); ">rows</span>(), <span style="color: rgb(77, 129, 134); ">AA</span>.<span style="color: rgb(48, 89, 93); ">columns</span>());<span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>);</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "><span style="color: rgb(0, 0, 0); "> </span>// get number of non-zeros in both diagonal and non-diagonal portions of the matrix</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(77, 129, 134); ">std</span>::<span style="color: rgb(77, 129, 134); ">pair</span><<span style="color: rgb(77, 129, 134); ">size_t</span>,<span style="color: rgb(77, 129, 134); ">size_t</span>> <span style="color: rgb(77, 129, 134); ">nz</span> = <span style="color: rgb(77, 129, 134); ">AA</span>.non_zero_off_diagonal();</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(77, 129, 134); ">ierr</span> = MatMPIAIJSetPreallocation(A_, <span style="color: rgb(77, 129, 134); ">nz</span>.<span style="color: rgb(77, 129, 134); ">first</span>, <span style="color: rgb(77, 129, 134); ">PETSC_NULL</span>, <span style="color: rgb(77, 129, 134); ">nz</span>.<span style="color: rgb(77, 129, 134); ">second</span>, <span style="color: rgb(77, 129, 134); ">PETSC_NULL</span>); <span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>);</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(77, 129, 134); ">ierr</span> = MatSeqAIJSetPreallocation(A_, <span style="color: rgb(77, 129, 134); ">nz</span>.<span style="color: rgb(77, 129, 134); ">first</span>, <span style="color: rgb(77, 129, 134); ">PETSC_NULL</span>); <span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>);</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); ">#ifdef VERBOSE</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> cout<<<span style="color: rgb(158, 32, 32); ">"Ok! "</span><<<span style="color: rgb(77, 129, 134); ">timer</span><<endl;</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); ">#endif</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> }</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(38, 126, 61); ">/* </span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "> Currently, all PETSc parallel matrix formats are partitioned by</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "> contiguous chunks of rows across the processors. Determine which</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "> rows of the matrix are locally owned. </div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "> */</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(77, 129, 134); ">ierr</span> = MatGetOwnershipRange(A_,&<span style="color: rgb(77, 129, 134); ">Istart</span>,&<span style="color: rgb(77, 129, 134); ">Iend</span>);<span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>);</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); ">#ifdef VERBOSE</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(158, 32, 32); "><span style="color: rgb(0, 0, 0); "> </span><span style="color: rgb(121, 72, 48); ">MPIPRINT</span><span style="color: rgb(0, 0, 0); ">(</span>" Got ownership range... "<span style="color: rgb(0, 0, 0); ">);</span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> cout<<<span style="color: rgb(77, 129, 134); ">timer</span><<endl;</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); ">#endif</div><p style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></p><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "><span style="color: rgb(0, 0, 0); "> </span>// zero-out matrix</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> MatZeroEntries(A_);</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); ">#ifdef VERBOSE</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(158, 32, 32); "><span style="color: rgb(0, 0, 0); "> </span><span style="color: rgb(121, 72, 48); ">MPIPRINT</span><span style="color: rgb(0, 0, 0); ">(</span>" Zeroed-out sparse matrix entries... "<span style="color: rgb(0, 0, 0); ">);</span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> cout<<<span style="color: rgb(77, 129, 134); ">timer</span><<endl;</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); ">#endif</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(77, 129, 134); "><span style="color: rgb(0, 0, 0); "> </span><span style="color: rgb(140, 34, 95); ">for</span><span style="color: rgb(0, 0, 0); "> (</span>sparse_matrix_type<span style="color: rgb(0, 0, 0); ">::</span>const_hash_iterator<span style="color: rgb(0, 0, 0); "> </span>it<span style="color: rgb(0, 0, 0); "> = </span>AA<span style="color: rgb(0, 0, 0); ">.</span>map_<span style="color: rgb(0, 0, 0); ">.</span><span style="color: rgb(48, 89, 93); ">begin</span><span style="color: rgb(0, 0, 0); ">(); </span>it<span style="color: rgb(0, 0, 0); "> != </span>AA<span style="color: rgb(0, 0, 0); ">.</span>map_<span style="color: rgb(0, 0, 0); ">.</span><span style="color: rgb(48, 89, 93); ">end</span><span style="color: rgb(0, 0, 0); ">(); ++</span>it<span style="color: rgb(0, 0, 0); ">) {</span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "><span style="color: rgb(0, 0, 0); "> </span>// get subscripts</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(77, 129, 134); "><span style="color: rgb(0, 0, 0); "> </span>std<span style="color: rgb(0, 0, 0); ">::</span>pair<span style="color: rgb(0, 0, 0); "><</span>size_t<span style="color: rgb(0, 0, 0); ">,</span>size_t<span style="color: rgb(0, 0, 0); ">> </span>subs<span style="color: rgb(0, 0, 0); "> = </span>AA<span style="color: rgb(0, 0, 0); ">.</span><span style="color: rgb(48, 89, 93); ">unhash</span><span style="color: rgb(0, 0, 0); ">(</span>it<span style="color: rgb(0, 0, 0); ">-></span>first<span style="color: rgb(0, 0, 0); ">);</span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(140, 34, 95); ">int</span> <span style="color: rgb(77, 129, 134); ">row</span> = <span style="color: rgb(77, 129, 134); ">subs</span>.<span style="color: rgb(77, 129, 134); ">first</span>;</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(140, 34, 95); ">int</span> <span style="color: rgb(77, 129, 134); ">col</span> = <span style="color: rgb(77, 129, 134); ">subs</span>.<span style="color: rgb(77, 129, 134); ">second</span>;</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(77, 129, 134); ">ierr</span> = MatSetValues(A_, <span style="color: rgb(6, 63, 244); ">1</span>, &<span style="color: rgb(77, 129, 134); ">row</span>, <span style="color: rgb(6, 63, 244); ">1</span>, &<span style="color: rgb(77, 129, 134); ">col</span>, &<span style="color: rgb(77, 129, 134); ">it</span>-><span style="color: rgb(77, 129, 134); ">second</span>, ADD_VALUES);<span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>);</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> }</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); ">#ifdef VERBOSE</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(158, 32, 32); "><span style="color: rgb(0, 0, 0); "> </span><span style="color: rgb(121, 72, 48); ">MPIPRINT</span><span style="color: rgb(0, 0, 0); ">(</span>" Filled sparse matrix... "<span style="color: rgb(0, 0, 0); ">);</span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> cout<<<span style="color: rgb(77, 129, 134); ">timer</span><<endl;</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); ">#endif</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(38, 126, 61); ">/* </span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "> Assemble matrix, using the 2-step process:</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "> MatAssemblyBegin(), MatAssemblyEnd()</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "> Computations can be done while messages are in transition</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "> by placing code between these two statements.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "> */</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(77, 129, 134); ">ierr</span> = MatAssemblyBegin(A_,MAT_FINAL_ASSEMBLY);<span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>);</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(77, 129, 134); ">ierr</span> = MatAssemblyEnd(A_,MAT_FINAL_ASSEMBLY);<span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>);</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); ">#ifdef VERBOSE</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(158, 32, 32); "><span style="color: rgb(0, 0, 0); "> </span><span style="color: rgb(121, 72, 48); ">MPIPRINT</span><span style="color: rgb(0, 0, 0); ">(</span>" Sparse matrix assembled... "<span style="color: rgb(0, 0, 0); ">);</span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> cout<<<span style="color: rgb(77, 129, 134); ">timer</span><<endl;</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); ">#endif</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(77, 129, 134); ">ierr</span> = MatSetOption(A_, MAT_NEW_NONZERO_LOCATIONS, PETSC_FALSE);<span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>);</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "><span class="Apple-style-span" style="color: rgb(0, 0, 0); "> </span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(38, 126, 61); ">/* </span><span class="Apple-style-span" style="color: rgb(38, 126, 61); ">Create parallel vectors. */</span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(77, 129, 134); ">ierr</span> = VecCreate(PETSC_COMM_WORLD,&<span style="color: rgb(77, 129, 134); ">b</span>);<span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>);</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(77, 129, 134); ">ierr</span> = VecSetSizes(<span style="color: rgb(77, 129, 134); ">b</span>,PETSC_DECIDE, <span style="color: rgb(77, 129, 134); ">bb</span>.<span style="color: rgb(48, 89, 93); ">size</span>());<span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>);</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(77, 129, 134); ">ierr</span> = VecSetFromOptions(<span style="color: rgb(77, 129, 134); ">b</span>);<span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>);</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(140, 34, 95); ">if</span> (!allocated_) {</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(77, 129, 134); ">ierr</span> = VecDuplicate(<span style="color: rgb(77, 129, 134); ">b</span>,&x_);<span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>); </div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "><span style="color: rgb(0, 0, 0); "> </span>// set flag to true</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> allocated_ = <span style="color: rgb(140, 34, 95); ">true</span>;</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> }</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> VecZeroEntries(x_); </div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); ">#ifdef VERBOSE</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(158, 32, 32); "><span style="color: rgb(0, 0, 0); "> </span><span style="color: rgb(121, 72, 48); ">MPIPRINT</span><span style="color: rgb(0, 0, 0); ">(</span>" Vectors created... "<span style="color: rgb(0, 0, 0); ">);</span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> cout<<<span style="color: rgb(77, 129, 134); ">timer</span><<endl;</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); ">#endif</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "><span style="color: rgb(0, 0, 0); "> </span>/* Set hight-hand-side vector */</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(77, 129, 134); "><span style="color: rgb(0, 0, 0); "> </span><span style="color: rgb(140, 34, 95); ">for</span><span style="color: rgb(0, 0, 0); "> (</span>sparse_vector_type<span style="color: rgb(0, 0, 0); ">::</span>const_hash_iterator<span style="color: rgb(0, 0, 0); "> </span>it<span style="color: rgb(0, 0, 0); "> = </span>bb<span style="color: rgb(0, 0, 0); ">.</span>map_<span style="color: rgb(0, 0, 0); ">.</span><span style="color: rgb(48, 89, 93); ">begin</span><span style="color: rgb(0, 0, 0); ">(); </span>it<span style="color: rgb(0, 0, 0); "> != </span>bb<span style="color: rgb(0, 0, 0); ">.</span>map_<span style="color: rgb(0, 0, 0); ">.</span><span style="color: rgb(48, 89, 93); ">end</span><span style="color: rgb(0, 0, 0); ">(); ++</span>it<span style="color: rgb(0, 0, 0); ">) {</span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(140, 34, 95); ">int</span> <span style="color: rgb(77, 129, 134); ">row</span> = <span style="color: rgb(77, 129, 134); ">it</span>-><span style="color: rgb(77, 129, 134); ">first</span>;</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(77, 129, 134); ">ierr</span> = VecSetValues(<span style="color: rgb(77, 129, 134); ">b</span>, <span style="color: rgb(6, 63, 244); ">1</span>, &<span style="color: rgb(77, 129, 134); ">row</span>, &<span style="color: rgb(77, 129, 134); ">it</span>-><span style="color: rgb(77, 129, 134); ">second</span>, ADD_VALUES);</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> }</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); ">#ifdef VERBOSE</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(158, 32, 32); "><span style="color: rgb(0, 0, 0); "> </span><span style="color: rgb(121, 72, 48); ">MPIPRINT</span><span style="color: rgb(0, 0, 0); ">(</span>" Right hand side set... "<span style="color: rgb(0, 0, 0); ">);</span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> cout<<<span style="color: rgb(77, 129, 134); ">timer</span><<endl;</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); ">#endif</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(38, 126, 61); ">/* </span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "> Assemble vector, using the 2-step process:</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "> VecAssemblyBegin(), VecAssemblyEnd()</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "> Computations can be done while messages are in transition</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "> by placing code between these two statements.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "> */</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(77, 129, 134); ">ierr</span> = VecAssemblyBegin(<span style="color: rgb(77, 129, 134); ">b</span>);<span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>);</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(77, 129, 134); ">ierr</span> = VecAssemblyEnd(<span style="color: rgb(77, 129, 134); ">b</span>);<span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>);</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); ">#ifdef VERBOSE</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(158, 32, 32); "><span style="color: rgb(0, 0, 0); "> </span><span style="color: rgb(121, 72, 48); ">MPIPRINT</span><span style="color: rgb(0, 0, 0); ">(</span>" Right-hand-side vector assembled... "<span style="color: rgb(0, 0, 0); ">);</span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> cout<<<span style="color: rgb(77, 129, 134); ">timer</span><<endl;</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); ">#endif</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "><span class="Apple-style-span" style="color: rgb(0, 0, 0); "> <span style="color: rgb(38, 126, 61); ">/* </span></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "> Set operators. Here the matrix that defines the linear system</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "> also serves as the preconditioning matrix.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "> */</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(77, 129, 134); ">ierr</span> = KSPSetOperators(ksp_,A_,A_,SAME_NONZERO_PATTERN);<span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>);</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "><span class="Apple-style-span" style="color: rgb(38, 126, 61); "><span style="color: rgb(0, 0, 0); "> </span>/* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - </span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "> Solve the linear system</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */<span class="Apple-style-span" style="color: rgb(0, 0, 0); "> </span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(77, 129, 134); ">ierr</span> = KSPSolve(ksp_,<span style="color: rgb(77, 129, 134); ">b</span>,x_);<span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>);</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); ">#ifdef VERBOSE</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(158, 32, 32); "><span style="color: rgb(0, 0, 0); "> </span><span style="color: rgb(121, 72, 48); ">MPIPRINT</span><span style="color: rgb(0, 0, 0); ">(</span>" System solved... "<span style="color: rgb(0, 0, 0); ">);</span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> cout<<<span style="color: rgb(77, 129, 134); ">timer</span><<endl;</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); ">#endif</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "><span style="color: rgb(0, 0, 0); "> </span>/* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - </div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "> Check solution and clean up</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */</div><p style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></p><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> VecScatter <span style="color: rgb(77, 129, 134); ">ctx</span>;</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> Vec <span style="color: rgb(77, 129, 134); ">x_all</span>;</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(77, 129, 134); ">ierr</span> = VecScatterCreateToAll(x_, &<span style="color: rgb(77, 129, 134); ">ctx</span>, &<span style="color: rgb(77, 129, 134); ">x_all</span>);<span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>);</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(77, 129, 134); ">ierr</span> = VecScatterBegin(<span style="color: rgb(77, 129, 134); ">ctx</span>,x_,<span style="color: rgb(77, 129, 134); ">x_all</span>,INSERT_VALUES,SCATTER_FORWARD);<span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>);</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(77, 129, 134); ">ierr</span> = VecScatterEnd(<span style="color: rgb(77, 129, 134); ">ctx</span>,x_,<span style="color: rgb(77, 129, 134); ">x_all</span>,INSERT_VALUES,SCATTER_FORWARD);<span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>);</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); ">#ifdef VERBOSE</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(158, 32, 32); "><span style="color: rgb(0, 0, 0); "> </span><span style="color: rgb(121, 72, 48); ">MPIPRINT</span><span style="color: rgb(0, 0, 0); ">(</span>" Solution vector scattered... "<span style="color: rgb(0, 0, 0); ">);</span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> cout<<<span style="color: rgb(77, 129, 134); ">timer</span><<endl;</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); ">#endif</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "><span style="color: rgb(0, 0, 0); "> </span>/* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - </div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "> Get values from solution and store them in the object that will be</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "> returned</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(77, 129, 134); "><span style="color: rgb(0, 0, 0); "> </span>sparse_vector_type<span style="color: rgb(0, 0, 0); "> </span>xx<span style="color: rgb(0, 0, 0); ">(</span>bb<span style="color: rgb(0, 0, 0); ">.</span><span style="color: rgb(48, 89, 93); ">size</span><span style="color: rgb(0, 0, 0); ">());</span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "><span style="color: rgb(0, 0, 0); "> </span>/* Set solution vector */</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(77, 129, 134); "><span style="color: rgb(0, 0, 0); "> </span><span style="color: rgb(140, 34, 95); ">for</span><span style="color: rgb(0, 0, 0); "> (</span>sparse_vector_type<span style="color: rgb(0, 0, 0); ">::</span>const_hash_iterator<span style="color: rgb(0, 0, 0); "> </span>it<span style="color: rgb(0, 0, 0); "> = </span>bb<span style="color: rgb(0, 0, 0); ">.</span>map_<span style="color: rgb(0, 0, 0); ">.</span><span style="color: rgb(48, 89, 93); ">begin</span><span style="color: rgb(0, 0, 0); ">(); </span>it<span style="color: rgb(0, 0, 0); "> != </span>bb<span style="color: rgb(0, 0, 0); ">.</span>map_<span style="color: rgb(0, 0, 0); ">.</span><span style="color: rgb(48, 89, 93); ">end</span><span style="color: rgb(0, 0, 0); ">(); ++</span>it<span style="color: rgb(0, 0, 0); ">) {</span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(140, 34, 95); ">int</span> <span style="color: rgb(77, 129, 134); ">row</span> = <span style="color: rgb(77, 129, 134); ">it</span>-><span style="color: rgb(77, 129, 134); ">first</span>;</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(77, 129, 134); ">ierr</span> = VecGetValues(<span style="color: rgb(77, 129, 134); ">x_all</span>, <span style="color: rgb(6, 63, 244); ">1</span>, &<span style="color: rgb(77, 129, 134); ">row</span>, &<span style="color: rgb(77, 129, 134); ">xx</span>[<span style="color: rgb(77, 129, 134); ">row</span>]);</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> }</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); ">#ifdef VERBOSE</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(158, 32, 32); "><span style="color: rgb(0, 0, 0); "> </span><span style="color: rgb(121, 72, 48); ">MPIPRINT</span><span style="color: rgb(0, 0, 0); ">(</span>" Solution vector copied... "<span style="color: rgb(0, 0, 0); ">);</span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> cout<<<span style="color: rgb(77, 129, 134); ">timer</span><<endl;</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); ">#endif</div><p style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></p><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <span style="color: rgb(77, 129, 134); ">ierr</span> = VecDestroy(<span style="color: rgb(77, 129, 134); ">b</span>);<span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>);</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(77, 129, 134); ">ierr</span> = VecDestroy(<span style="color: rgb(77, 129, 134); ">x_all</span>);<span style="color: rgb(121, 72, 48); ">CHKERR</span>(<span style="color: rgb(77, 129, 134); ">ierr</span>);</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); ">#ifdef VERBOSE</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(158, 32, 32); "><span style="color: rgb(0, 0, 0); "> </span><span style="color: rgb(121, 72, 48); ">MPIPRINT</span><span style="color: rgb(0, 0, 0); ">(</span>" Temporary data structures destroyed... "<span style="color: rgb(0, 0, 0); ">);</span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> cout<<<span style="color: rgb(77, 129, 134); ">timer</span><<endl;</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(125, 72, 49); ">#endif</div><p style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; min-height: 21px; "> <br class="webkit-block-placeholder"></p><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: rgb(140, 34, 95); ">return</span> <span style="color: rgb(77, 129, 134); ">xx</span>;</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; ">}</div></div><div><br></div></body></html>