<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">Marius:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div style="font-family:Verdana;font-size:12px"><div>Thanks a lot, I did not test it thoroughly but it seems to work well and really helpful. I have one question, is it necessary to do the MatTranspose step as in ex214.c, I thought this is handled internally by petsc?</div></div></div></blockquote><div><br></div><div>MUMPS requires sparse compressed COLUMN format in the host for rhs matrix, which is not supported by PETSc. <span style="font-family:Verdana;font-size:12px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">MatTranspose() is expensive for large size matrices.</span></div><div>We added a new petsc function </div><div>MatMatTransposeSolve(): Solves A X = B^T, A: factored matrix</div><div>(see petsc/src/mat/examples/tests/ex214c)</div><div><br></div><div>With this function, user does not need to call <span style="font-family:Verdana;font-size:12px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">MatTranspose, but needs to create B^T in</span></div><div><font face="Verdana"><span style="font-size:12px">AIJ matrix with all rows and columns stored in processor[0].</span></font></div><div><font face="Verdana"><span style="font-size:12px">Let us know if any of you have better idea to handle it.</span></font></div><div><font face="Verdana"><span style="font-size:12px">The easiest way for us is MUMPS supports sparse <span style="font-family:arial,sans-serif;font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">compressed<span> row format for rhs matrix.</span></span></span></font></div><div><font face="Verdana"><span style="font-size:12px"><span style="font-family:arial,sans-serif;font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span><br></span></span></span></font></div><div><font face="Verdana"><span style="font-size:12px"><span style="font-family:arial,sans-serif;font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span>The branch </span></span></span></font>hzhang/mumps-spRHS will be merged to petsc-master soon.</div><div><font face="Verdana"><span style="font-size:12px"><span style="font-family:arial,sans-serif;font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span><br></span></span></span></font></div><div>Hong</div><div><font face="Verdana"><span style="font-size:12px"><br></span></font></div><div><font face="Verdana"><span style="font-size:12px"><br></span></font></div><div><font face="Verdana"><span style="font-size:12px"><br></span></font></div><div><font face="Verdana"><span style="font-size:12px"><br></span></font></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div style="font-family:Verdana;font-size:12px"><div><div class="gmail-h5">
<div> </div>
<div> </div>
<div>
<div name="quote" style="margin:10px 5px 5px 10px;padding:10px 0px 10px 10px;border-left:2px solid rgb(195,217,229);word-wrap:break-word">
<div style="margin:0px 0px 10px"> </div>
<div name="quoted-content">
<div><span style="font-size:12.8px;float:none;display:inline">Marius:</span>
<div><span style="font-size:12.8px">I added support for parallel sparse RHS (in host) for MatMatSolve_MUMPS()</span></div>
<div><span style="font-size:12.8px"><a href="https://bitbucket.org/petsc/petsc/commits/2b691707dd0cf456c808def006e14b6f56b364b6" target="_blank">https://bitbucket.org/petsc/<wbr>petsc/commits/<wbr>2b691707dd0cf456c808def006e14b<wbr>6f56b364b6</a></span></div>
<div> </div>
<div><span style="font-size:12.8px">It is in the branch hzhang/mumps-spRHS.</span></div>
<div><span style="font-size:12.8px">You may test it.</span></div>
<div> </div>
<div><span style="font-size:12.8px">I'll further cleanup the routine, test it, then merge it to petsc.</span></div>
<div><span style="font-size:12.8px">Hong</span>
<div class="gmail_extra">
<div class="gmail_quote">On Mon, Jun 4, 2018 at 9:54 AM, Hong <span><<a href="mailto:hzhang@mcs.anl.gov" target="_blank">hzhang@mcs.anl.gov</a>></span> wrote:
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div class="gmail_extra">
<div class="gmail_quote"><span class="gmail-m_5941660939760593678gmail-">On Mon, Jun 4, 2018 at 1:03 PM, Jean-Yves LExcellent <span><<a href="mailto:Jean-Yves.L.Excellent@ens-lyon.fr" target="_blank">Jean-Yves.L.Excellent@ens-<wbr>lyon.fr</a>></span> wrote:</span>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093moz-cite-prefix">
<div class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093moz-cite-prefix"><br>
<span class="gmail-m_5941660939760593678gmail-">Thanks for the details of your needs.<br>
<br>
For the first application, the sparse RHS feature with distributed<br>
solution should effectively be fine.</span></div>
</div>
</div>
</blockquote>
<div>I'll add parallel support of this feature in petsc after I'm back the ANL after June 14th.</div>
<div><span class="gmail-m_5941660939760593678gmail-HOEnZb"><font color="#888888">Hong</font></span></div>
<div>
<div class="gmail-m_5941660939760593678gmail-h5">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093moz-cite-prefix">
<div class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093moz-cite-prefix"><br>
For the second one, a future distributed RHS feature (not currently<br>
available in MUMPS) might help if the centralized sparse RHS is too<br>
memory consuming, depending on the size of B1.<br>
<br>
Regards,<br>
Jean-Yves, for the MUMPS developers<br>
<br>
</div>
<blockquote>
<div style="font-family:Verdana;font-size:12px">
<div><span> I want to invert a rather large sparse matrix for this using a sparse rhs with centralized input would be ok as long as the solution is distributed. </span></div>
</div>
</blockquote>
</div>
<blockquote>
<div style="font-family:Verdana;font-size:12px">
<div>and the second application I have in mind is solving a system of the from AX=B where A and B are sparse and B is given by a block matrix of the form B=[B1 0, 0 0] where B1 is dense but the dimension is (much) smaller than that of the whole matrix B.
<div>
<div class="gmail-m_5941660939760593678gmail-m_-3820402716620242094h5">
<div style="margin:10px 5px 5px 10px;padding:10px 0px 10px 10px;border-left:2px solid rgb(195,217,229)">
<div style="margin:0px 0px 10px"> </div>
<div>
<div>
<div class="gmail_extra">
<div class="gmail_quote">Marius:</div>
<div class="gmail_quote">Current PETSc interface supports sequential sparse <span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline">multiple right-hand side, but not distributed.</span></div>
<div class="gmail_quote">It turns out that mumps does not support <span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline">distributed sparse multiple right-hand sides at</span><br style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
<span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline">the moment (see attached email). </span></div>
<div class="gmail_quote"> </div>
<div class="gmail_quote"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline">Jean-Yves invites you to communicate with him directly. </span></span></div>
<div class="gmail_quote"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline">Let me know what we can help on this matter, </span></span></div>
<div class="gmail_quote"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline">e.g., add support for parallel implementation of <span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline">sparse </span><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline">multiple right-hand side with </span></span></span></div>
<div class="gmail_quote"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline">centralized rhs input?</span></span></span></div>
<div class="gmail_quote"> </div>
<div class="gmail_quote"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline">Hong</span></span></div>
<div class="gmail_quote"> </div>
<div class="gmail_quote"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline">----------------------</span></div>
<div class="gmail_quote">
<div class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-gE gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-iv gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-gt" style="padding:10px 0px 3px;font-size:12.8px;color:rgb(34,34,34);font-family:arial,sans-serif;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
<table cellpadding="0" class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-cf gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-gJ" style="border-collapse:collapse;margin-top:0px;width:auto">
<tbody>
<tr class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-acZ" style="height:16px">
<td class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-gF gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-gK" style="font-family:arial,sans-serif;margin:0px;text-align:left;white-space:nowrap;padding-right:8px;vertical-align:top;width:270.667px;padding-top:0px">
<table cellpadding="0" class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-cf gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-ix" style="border-collapse:collapse;table-layout:fixed;width:270.667px">
<tbody>
<tr>
<td style="font-family:arial,sans-serif;margin:0px">
<h3 class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-iw" style="overflow:hidden;white-space:nowrap;max-width:92%;display:inline-block;font-size:inherit;font-weight:inherit;margin:inherit"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><span class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-gD" style="color:rgb(34,34,34);font-size:12.8px;font-weight:bold;white-space:nowrap;display:inline;vertical-align:top">Jean-Yves LExcellent</span><span> </span><span class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-go" style="vertical-align:top;color:rgb(85,85,85)"><span><</span><a href="mailto:Jean-Yves.L.Excellent@ens-lyon.fr" target="_blank">Jean-Yves.L.<wbr>Excellent@ens-lyon.fr</a><span>></span></span></span></h3>
</td>
</tr>
</tbody>
</table>
</td>
<td class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-gH gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-bAk" style="font-family:arial,sans-serif;margin:0px;text-align:right;white-space:nowrap;vertical-align:top;color:rgb(34,34,34)">
<div class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-gK" style="padding-top:0px;padding-right:2px"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><span class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-g3" id="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-:pm" style="vertical-align:top;margin-right:3px" title="Fri, Jun 1, 2018 at 5:14 AM">5:14 AM (3 hours ago)</span></span>
<div class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-zd gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-bi4" style="display:inline-block;height:20px;outline:0px" title="Not
starred"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><span class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-KT" style="display:inline-block;height:19px;text-align:center;width:19px;padding:2px"><img alt="" class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-f gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-KT-JX" src="https://mail.google.com/mail/u/0/images/cleardot.gif" style="margin-top: 0px; vertical-align: top;"></span></span></div>
</div>
</td>
<td class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-gH" style="font-family:arial,sans-serif;margin:0px;text-align:right;white-space:nowrap;vertical-align:top;color:rgb(34,34,34)"> </td>
<td class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-gH gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-acX gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-bAm" rowspan="2" style="font-family:arial,sans-serif;margin:0px;text-align:right;white-space:nowrap;vertical-align:top;color:rgb(34,34,34)">
<div class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-I gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-J-J5-Ji gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-I-Js-IF gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-aaq gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-I-ax7 gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-L3" style="display:inline-block;font-size:11px;font-weight:bold;text-align:center;white-space:nowrap;margin-right:0px;height:27px;line-height:27px;min-width:32px;outline:0px;padding:0px 8px;background-color:rgb(245,245,245);color:rgb(68,68,68)"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><img alt="" class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-hB gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-I-J3" src="https://mail.google.com/mail/u/0/images/cleardot.gif" style="height: 21px; vertical-align: middle; width: 21px;"></span></div>
<div class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-I gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-J-J5-Ji gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-I-Js-Gs gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-aap gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-I-awG gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-I-ax7 gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-L3" id="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-:px" style="display:inline-block;font-size:11px;font-weight:bold;text-align:center;white-space:nowrap;margin-right:0px;height:27px;line-height:27px;min-width:21px;outline:0px;padding:0px;background-color:rgb(245,245,245);color:rgb(68,68,68)"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><img alt="" class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-hA gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-I-J3" src="https://mail.google.com/mail/u/0/images/cleardot.gif" style="height: 21px; width: 21px; vertical-align: middle;"></span></div>
</td>
</tr>
<tr class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-acZ gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-xD" style="height:16px">
<td colspan="3" style="font-family:arial,sans-serif;margin:0px">
<table cellpadding="0" class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-cf gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-adz" style="border-collapse:collapse;table-layout:fixed;white-space:nowrap;width:434.667px">
<tbody>
<tr>
<td class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-ady" style="font-family:arial,sans-serif;margin:0px;overflow:hidden;white-space:nowrap">
<div class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-iw gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-ajw" style="overflow:hidden;white-space:nowrap;max-width:92%;display:inline-block"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><span class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-hb" style="vertical-align:top;color:rgb(119,119,119)">to<span> </span><span class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-g2">Hong</span>,<span> </span><span class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-g2">mumps-dev</span></span></span></div>
<div class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-ajy" style="display:inline-block;margin-left:5px;vertical-align:top"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><img alt="" class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-ajz" id="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-:pw" src="https://mail.google.com/mail/u/0/images/cleardot.gif" style="padding: 0px 0px 1px; vertical-align: bottom; height: 12px; width: 12px;"></span></div>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</div>
<div id="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-:pi" style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:medium;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
<div class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-qQVYZb"> </div>
<div class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-utdU2e"> </div>
<div class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-btm"> </div>
</div>
<div class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-" style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:medium;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
<div class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-aHl"> </div>
<div class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-ajV" id="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-:pv" style="outline:none;padding:5px 0px;width:22px;margin:0px">
<div class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-ajR" id="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-:qe" style="background-color:rgb(241,241,241);border:1px solid rgb(221,221,221);clear:both;line-height:6px;outline:none;width:20px"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><img class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-ajT" src="https://ssl.gstatic.com/ui/v1/icons/mail/images/cleardot.gif" style="height: 8px; width: 20px;"></span></div>
</div>
<div class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-ii gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-gt gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-adP" id="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-:pk" style="font-size:12.8px;direction:ltr;margin:5px 15px 0px 0px;padding-bottom:5px">
<div class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-a3s gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-aXjCH gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-m163bad8b399eea70" id="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093gmail-:pj" style="overflow:hidden"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline">Hello,<br>
<br>
We do not support distributed sparse multiple right-hand sides at<br>
the moment. From the feedback we have from applications, the<br>
right-hand sides are often very sparse, and having them distributed<br>
did not seem critical.<br>
<br>
Since we are specifying a distributed right-hand sides feature at the<br>
moment, could you let us know in more detail the need regarding<br>
distributed sparse right-hand side (e.g., do all columns have the same<br>
nonzero structure in that case) or put us in contact with the user who<br>
needs this?<br>
<br>
Thanks,<br>
Jean-Yves and Patrick</span></div>
</div>
</div>
</div>
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Thanks a lot guys, very helpful.<br>
<span> <br>
<br>
<br>
<br>
I see MUMPS <a href="http://mumps.enseeiht.fr/" target="_blank">http://mumps.enseeiht.<wbr>fr/</a></span><br>
<br>
Sparse multiple right-hand side, distributed solution; Exploitation of sparsity in the right-hand sidesPETSc interface computes mumps distributed solution as default (this is not new) (ICNTL(21) = 1)<br>
<span> <br>
I will add support for Sparse multiple right-hand side.<br>
<br>
Hong<br>
</span><br>
<span>On Thu, May 31, 2018 at 11:25 AM, Smith, Barry F. <<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>[mailto:<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsm<wbr>ith@mcs.anl.gov</a>]> wrote:<br>
Hong,<br>
<br>
Can you see about adding support for distributed right hand side?<br>
<br>
Thanks<br>
<br>
Barry</span><br>
<br>
<span>> On May 31, 2018, at 2:37 AM, Marius Buerkle <<a href="mailto:mbuerkle@web.de" target="_blank">mbuerkle@web.de</a>[mailto:<a href="mailto:mbuerkle@web.de" target="_blank">mbuerk<wbr>le@web.de</a>]> wrote:<br>
><br>
> The fix for MAT_NEW_NONZERO_LOCATIONS, thanks again.<br>
><br>
> I have yet another question, sorry. The recent version of MUMPS supports distributed and sparse RHS is there any chance that this will be supported in PETSc in the near future?<br>
><br>
> <br>
> <br>
></span><br>
<span>>> On May 30, 2018, at 6:55 PM, Marius Buerkle <<a href="mailto:mbuerkle@web.de" target="_blank">mbuerkle@web.de</a>[mailto:<a href="mailto:mbuerkle@web.de" target="_blank">mbuerk<wbr>le@web.de</a>]> wrote:<br>
>><br>
>> Thanks for the quick fix, I will test it and report back.<br>
>> I have another maybe related question, if MAT_NEW_NONZERO_LOCATIONS is true and let's say 1 new nonzero position is created it does not allocated 1 but several new nonzeros but only use 1.<br>
><br>
> Correct<br>
><br>
>> I think that is normal, right?<br>
><br>
> Yes<br>
><br>
>> But, at least as far as I understand the manual, a subsequent call of mat assemble with<br>
>> MAT_FINAL_ASSEMBLY should compress out the unused allocations and release the memory, is this correct?<br>
><br>
> It "compresses it out" (by shifting all the nonzero entries to the beginning of the internal i, j, and a arrays), but does NOT release any memory. Since the values are stored in one big contiguous array (obtained with a single malloc) it cannot just free part of the array, so the extra locations just sit harmlessly at the end if the array unused.<br>
><br>
>> If so, this did not work for me, even after doing<br>
>> MAT_FINAL_ASSEMBLY the unused nonzero allocations remain. Is this normal?<br>
><br>
> Yes,<br>
><br>
> Barry<br>
><br>
>><br>
>>><br>
>>> Fixed in the branch barry/fix-mat-new-nonzero-<wbr>locations/maint<br>
>>><br>
>>> Once this passes testing it will go into the maint branch and then the next patch release but you can use it now in the branch barry/fix-mat-new-nonzero-<wbr>locations/maint<br>
>>><br>
>>> Thanks for the report and reproducible example<br>
>>><br>
>>> Barry<br>
>>><br>
>>></span>
<div>
<div class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093h5">>>>> On May 29, 2018, at 7:51 PM, Marius Buerkle <<a href="mailto:mbuerkle@web.de" target="_blank">mbuerkle@web.de</a>[mailto:<a href="mailto:mbuerkle@web.de" target="_blank">mbuerk<wbr>le@web.de</a>]> wrote:<br>
>>>><br>
>>>> Sure, I made a small reproducer, it is Fortran though I hope that is ok. If MAT_NEW_NONZERO_LOCATIONS is set to false I get an error, if it is set to true the new nonzero element is inserted, if MAT_NEW_NONZERO_LOCATIONS is false and either MAT_NEW_NONZERO_LOCATION_ERR or MAT_NEW_NONZERO_ALLOCATION_ERR is set to false afterwards then the new nonzero is also created without an error, but if MAT_NEW_NONZERO_LOCATIONS is set to false after MAT_NEW_NONZERO_LOCATION_ERR/<wbr>MAT_NEW_NONZERO_ALLOCATION_ERR have been set to false I get an error again.<br>
>>>><br>
>>>><br>
>>>> program newnonzero<br>
>>>> #include <petsc/finclude/petscmat.h><br>
>>>> use petscmat<br>
>>>> implicit none<br>
>>>><br>
>>>> Mat :: A<br>
>>>> PetscInt :: dnnz,onnz,n,m,idxm(1),idxn(1),<wbr>nl1,nl2<br>
>>>> PetscScalar :: v(1)<br>
>>>> PetscReal :: info(MAT_INFO_SIZE)<br>
>>>> PetscErrorCode :: ierr<br>
>>>><br>
>>>> integer :: nproc,iproc,i<br>
>>>><br>
>>>> call PetscInitialize(PETSC_NULL_<wbr>CHARACTER,ierr)<br>
>>>><br>
>>>> call MPI_COMM_SIZE(PETSC_COMM_<wbr>WORLD, nproc,ierr)<br>
>>>><br>
>>>> call MPI_Comm_rank( PETSC_COMM_WORLD, iproc, ierr )<br>
>>>><br>
>>>> n=3<br>
>>>> m=n<br>
>>>> call MatCreateAIJ(PETSC_COMM_WORLD,<wbr>PETSC_DECIDE,PETSC_DECIDE,n,m,<wbr>1,PETSC_NULL_INTEGER,0,PETSC_<wbr>NULL_INTEGER,A,ierr)<br>
>>>><br>
>>>><br>
>>>> call MatGetOwnershipRange(A,nl1,<wbr>nl2,ierr)<br>
>>>> do i=nl1,nl2-1<br>
>>>> idxn(1)=i<br>
>>>> idxm(1)=i<br>
>>>> v(1)=1d0<br>
>>>> call MatSetValues(A,1,idxn,1,idxm, v,INSERT_VALUES,ierr)<br>
>>>> end do<br>
>>>> call MatAssemblyBegin(A,MAT_FINAL_<wbr>ASSEMBLY,ierr)<br>
>>>> call MatAssemblyEnd(A,MAT_FINAL_<wbr>ASSEMBLY,ierr)<br>
>>>><br>
>>>> call MatSetOption(A,MAT_NEW_<wbr>NONZERO_LOCATIONS,PETSC_FALSE,<wbr>ierr)<br>
>>>> !~ call MatSetOption(A,MAT_NEW_<wbr>NONZERO_LOCATION_ERR,PETSC_<wbr>FALSE,ierr)<br>
>>>> !~ call MatSetOption(A,MAT_NEW_<wbr>NONZERO_ALLOCATION_ERR ,PETSC_FALSE,ierr)<br>
>>>> !~ call MatSetOption(A,MAT_NEW_<wbr>NONZERO_LOCATIONS,PETSC_FALSE,<wbr>ierr)<br>
>>>><br>
>>>><br>
>>>> idxn(1)=0<br>
>>>> idxm(1)=n-1<br>
>>>> if ((idxn(1).ge.nl1).and.(idxn(1)<wbr>.le.nl2-1)) then<br>
>>>> v(1)=2d0<br>
>>>> call MatSetValues(A,1,idxn,1,idxm, v,INSERT_VALUES,ierr)<br>
>>>> end if<br>
>>>> call MatAssemblyBegin(A,MAT_FINAL_<wbr>ASSEMBLY,ierr)<br>
>>>> call MatAssemblyEnd(A,MAT_FINAL_<wbr>ASSEMBLY,ierr)<br>
>>>><br>
>>>> if ((idxn(1).ge.nl1).and.(idxn(1)<wbr>.le.nl2-1)) then<br>
>>>> v(1)=2d0<br>
>>>> call MatGetValues(A,1,idxn,1,idxm, v,ierr)<br>
>>>> write(6,*) v<br>
>>>> end if<br>
>>>><br>
>>>> call PetscFinalize(ierr)<br>
>>>><br>
>>>> end program newnonzero<br>
>>>><br>
>>>><br>
>>>><br>
>>>> $ mpiexec.hydra -n 3 ./a.out<br>
>>>> [0]PETSC ERROR: --------------------- Error Message ------------------------------<wbr>------------------------------<wbr>--<br>
>>>> [0]PETSC ERROR: Argument out of range<br>
>>>> [0]PETSC ERROR: Inserting a new nonzero at global row/column (0, 2) into matrix</div>
</div>
>>>> [0]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html%5Bhttp://www.mcs.anl.gov/petsc/documentation/faq.html%5D" target="_blank">http://www.mcs.anl.gov/petsc/<wbr>documentation/faq.html[http://<wbr>www.mcs.anl.gov/petsc/<wbr>documentation/faq.html]</a> for trouble shooting.<br>
<span class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093im gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093HOEnZb">>>>> [0]PETSC ERROR: Petsc Release Version 3.9.2, May, 20, 2018<br>
>>>> [0]PETSC ERROR: ./a.out on a named tono-hpc1 by marius Wed May 30 09:42:40 2018<br>
>>>> [0]PETSC ERROR: Configure options --prefix=/home/marius/prog/<wbr>petsc/3.9.2 --download-elemental=yes --download-metis=yes --download-parmetis=yes --download-mumps=yes --with-scalapack-lib="/home/<wbr>marius/intel/compilers_and_<wbr>libraries_2018.2.199/linux/<wbr>mkl/lib/intel64/libmkl_<wbr>scalapack_lp64.a -Wl,--start-group /home/marius/intel/compilers_<wbr>and_libraries_2018.2.199/<wbr>linux/mkl/lib/intel64/libmkl_<wbr>intel_lp64.a /home/marius/intel/compilers_<wbr>and_libraries_2018.2.199/<wbr>linux/mkl/lib/intel64/libmkl_<wbr>sequential.a /home/marius/intel/compilers_<wbr>and_libraries_2018.2.199/<wbr>linux/mkl/lib/intel64/libmkl_<wbr>core.a /home/marius/intel/compilers_<wbr>and_libraries_2018.2.199/<wbr>linux/mkl/lib/intel64/libmkl_<wbr>blacs_intelmpi_lp64.a -Wl,--end-group -lpthread -lm -ldl" --FC=mpiifort --CC=mpicc --CXX=mpicxx --with-scalar-type=complex --with-mpi-dir= --with-blaslapack-lib="/home/<wbr>marius/intel/compilers_and_<wbr>libraries_2018.2.199/linux/<wbr>mkl/lib/intel64/libmkl_<wbr>scalapack_lp64.a -Wl,--start-group /home/marius/intel/compilers_<wbr>and_libraries_2018.2.199/<wbr>linux/mkl/lib/intel64/libmkl_<wbr>intel_lp64.a /home/marius/intel/compilers_<wbr>and_libraries_2018.2.199/<wbr>linux/mkl/lib/intel64/libmkl_<wbr>sequential.a /home/marius/intel/compilers_<wbr>and_libraries_2018.2.199/<wbr>linux/mkl/lib/intel64/libmkl_<wbr>core.a /home/marius/intel/compilers_<wbr>and_libraries_2018.2.199/<wbr>linux/mkl/lib/intel64/libmkl_<wbr>blacs_intelmpi_lp64.a -Wl,--end-group -lpthread -lm -ldl" --with-cxx-dialect=C++11 --download-superlu_dist=yes --download-ptscotch=yes --with-x --with-debugging=1 --download-superlu=yes --with-mkl_cpardiso=1 --with-mkl_pardiso=1 --with-scalapack=1<br>
>>>> [0]PETSC ERROR: #1 MatSetValues_MPIAIJ() line 607 in /home/marius/prog/petsc/petsc-<wbr>3.9.2/src/mat/impls/aij/mpi/<wbr>mpiaij.c<br>
>>>> [0]PETSC ERROR: #2 MatSetValues() line 1312 in /home/marius/prog/petsc/petsc-<wbr>3.9.2/src/mat/interface/<wbr>matrix.c<br>
>>>> (0.000000000000000E+000,0.<wbr>000000000000000E+000)<br>
>>>><br>
>>>><br>
>>>><br>
>>>> Please send complete error message; type of matrix used etc. Ideally code that demonstrates the problem.<br>
>>>><br>
>>>> Barry<br>
>>>><br>
>>>></span>
<div class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093HOEnZb">
<div class="gmail-m_5941660939760593678gmail-m_-3820402716620242094m_-1671753951163694093h5">>>>>> On May 29, 2018, at 3:31 AM, Marius Buerkle <<a href="mailto:mbuerkle@web.de" target="_blank">mbuerkle@web.de</a>[mailto:<a href="mailto:mbuerkle@web.de" target="_blank">mbuerk<wbr>le@web.de</a>]> wrote:<br>
>>>>><br>
>>>>><br>
>>>>> Hi,<br>
>>>>><br>
>>>>> I tried to set MAT_NEW_NONZERO_LOCATIONS to false, as far as I understood MatSetValues should simply ignore entries which would give rise to new nonzero values not creating a new entry and not cause an error, but I get "[1]PETSC ERROR: Inserting a new nonzero at global row/column". Is this option supposed to work or not?<br>
>>>><br>
>>><br>
>>><br>
> <br>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<p> </p>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</div>
</div></div></div></div></div>
</blockquote></div><br></div></div>