<div dir="ltr"><div><div><div><div><div><div><div>An alternative would be this.<br>Assuming you can map each sub-domain row,col index into a global index for your coupled problem, you could<br><br></div>You create your matrix of type MATMPIAIJ<br></div><font>Then call</font><span style="font-family:arial,helvetica,sans-serif"><font><br>MatSeqAIJGetArray</font></span>()<br></div><span style="font-family:arial,helvetica,sans-serif"><font>MatGetRowIJ()<br></font></span></div><span style="font-family:arial,helvetica,sans-serif"><font>Then for each row of your sub-domain matrix,<br> 1) convert your local row index into a global row index<br> 2) traverse through the CSR array and copy the local column indices into a temporary array<br></font></span></div><span style="font-family:arial,helvetica,sans-serif"><font> 3) convert the local indices in the temporary array into global column indices<br></font></span></div><span style="font-family:arial,helvetica,sans-serif"><font> 4) Call MatSetValues() (with ADD_VALUES) and add the entries from each sub-domain into the parallel MPIAIJ matrix for the current row. You won't need to copy the aij entries, just index pointer returned by </font></span><span style="font-family:arial,helvetica,sans-serif"><font><span style="font-family:arial,helvetica,sans-serif"><font>MatSeqAIJGetArray() </font></span>so that it is points to the beginning of the entries for the current row.<br><br></font></span></div><span style="font-family:arial,helvetica,sans-serif"><font>Call the appropriate restore functions </font></span><br><span style="font-family:arial,helvetica,sans-serif"><font>MatRestoreRowIJ()<br></font></span><span style="font-family:arial,helvetica,sans-serif"><font>MatSeqAIJRestoreArray</font></span>()<br><div><span style="font-family:arial,helvetica,sans-serif"><font><br></font></span><div><div><div><span style="font-family:arial,helvetica,sans-serif"><font><br><br></font></span><div><div>PETSc team, seems the webpages<br> <a href="http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatSeqAIJGetArray.html">http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatSeqAIJGetArray.html</a><br></div><div> <a href="http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatSeqAIJRestoreArray.html">http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatSeqAIJRestoreArray.html</a><br>both contain a typo. The input matrix should be of type MATSEQAIJ<br></div><div><br></div><div>Cheers,<br></div><div> Dave<br></div><div><br><br></div></div></div></div></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On 23 November 2014 at 00:17, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class="">On Sat, Nov 22, 2014 at 4:30 PM, Alp Kalpalp <span dir="ltr"><<a href="mailto:alpkalpalp@gmail.com" target="_blank">alpkalpalp@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><div>Hi,<br><br></div>I am a newby in PetSc but I could not find a solution to my problem. Let me describbe it to you.I have an SeqAIJ matrix on each processor. These substructures stiffness matrices should be assembled on a MPIAIJ and they have overlapping dofs (on boundaries). A is Eigen::SparseMatrix<double> and my code was as follows;<br><br>ierr = MatCreateSeqAIJWithArrays(PETSC_COMM_SELF,m,n,A.outerIndexPtr(),A.innerIndexPtr(),A.valuePtr(),&subK);CHKERRQ(ierr);<br>ierr = MatCreateMPIAIJSumSeqAIJ(PETSC_COMM_WORLD,subK,PETSC_DECIDE,PETSC_DECIDE,MAT_INITIAL_MATRIX,&K);CHKERRQ(ierr);<br><br></div>Suppose I have 2 procs. first have 18x18 sparse mat, second hase 12x12 sparse matrix. Assembled system should be 24x24 for this specific problem. Above code is wenting to infinite loops. debugger reports that there is a deadlock on the progman. <br></div></div></blockquote><div><br></div></span><div>This function does not do this job:</div><div><br></div><div> <a href="http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatCreateMPIAIJSumSeqAIJ.html" target="_blank">http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatCreateMPIAIJSumSeqAIJ.html</a></div><div><br></div><div>It clearly says "The dimensions of the sequential matrix in each processor MUST be the same." <br></div><span class=""><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div></div><div>So, I searched for a workaroud and I found:<br><br>MatCreateMPIAIJWithArrays and this function seems fails when the matrices has overlapping parts. So that it asembles 30x30 matrix'!!<br><br></div><div>I check MatCreateMPIAIJ and it seems so costly to form d_nnz etc.<br></div><div><br>Finally, I tried the MatCreate, MatSetValues procedure, however in here MatSetValues requires a coordinate list of values not CSR format arrays!!!<br><br></div><div>So my question is how may I set and MPIAIJ from my (distributed) overlapping SeqAIJ matrices or preferebly using my CSR arrays directly?<br></div></div></blockquote><div><br></div></span><div>1) I think the best option is to put your element matrices directly into the MPIAIJ matrix, since each elemMat can be set with</div><div> a single call to MatSetValues().</div><div><br></div><div>2) If that is too much reorganization, then I would</div><div><br></div><div> a) Use MatCreateMPIAIJWithArrays() for the non-overlapping rows (remove the ghost rows)</div><div><br></div><div> b) Call MatSetValues() on each ghost row</div><div><br></div><div>3) If you cannot remove the overlapping rows, I would just call MatSetValues() on each row. It is unlikely that</div><div> the insertion process is a bottleneck in your computation (unless you are not preallocating correctly)</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div></div><div>Thanks in advance<br></div><div><br><div><br></div></div></div><span class="HOEnZb"><font color="#888888">
</font></span></blockquote></div><span class="HOEnZb"><font color="#888888"><br><br clear="all"><div><br></div>-- <br><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</font></span></div></div>
</blockquote></div><br></div>