<html><head></head><body><div style="font-family: Verdana;font-size: 12.0px;"><div>Thanks a lot, I did not test it thoroughly but it seems to work well and really helpful. I have one question, is it necessary to do the MatTranspose step as in ex214.c, I thought this is handled internally by petsc?</div>
<div> </div>
<div> </div>
<div>
<div name="quote" style="margin:10px 5px 5px 10px; padding: 10px 0 10px 10px; border-left:2px solid #C3D9E5; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div style="margin:0 0 10px 0;"> </div>
<div name="quoted-content">
<div><span style="font-size: 12.8px;float: none;display: inline;">Marius:</span>
<div><span style="font-size: 12.8px;">I added support for parallel sparse RHS (in host) for MatMatSolve_MUMPS()</span></div>
<div><span style="font-size: 12.8px;"><a href="https://bitbucket.org/petsc/petsc/commits/2b691707dd0cf456c808def006e14b6f56b364b6" target="_blank">https://bitbucket.org/petsc/petsc/commits/2b691707dd0cf456c808def006e14b6f56b364b6</a></span></div>
<div> </div>
<div><span style="font-size: 12.8px;">It is in the branch hzhang/mumps-spRHS.</span></div>
<div><span style="font-size: 12.8px;">You may test it.</span></div>
<div> </div>
<div><span style="font-size: 12.8px;">I'll further cleanup the routine, test it, then merge it to petsc.</span></div>
<div><span style="font-size: 12.8px;">Hong</span>
<div class="gmail_extra">
<div class="gmail_quote">On Mon, Jun 4, 2018 at 9:54 AM, Hong <span><<a href="mailto:hzhang@mcs.anl.gov" onclick="parent.window.location.href='hzhang@mcs.anl.gov'; return false;" target="_blank">hzhang@mcs.anl.gov</a>></span> wrote:
<blockquote class="gmail_quote" style="margin: 0.0px 0.0px 0.0px 0.8ex;border-left: 1.0px solid rgb(204,204,204);padding-left: 1.0ex;">
<div>
<div class="gmail_extra">
<div class="gmail_quote"><span class="gmail-">On Mon, Jun 4, 2018 at 1:03 PM, Jean-Yves LExcellent <span><<a href="mailto:Jean-Yves.L.Excellent@ens-lyon.fr" onclick="parent.window.location.href='Jean-Yves.L.Excellent@ens-lyon.fr'; return false;" target="_blank">Jean-Yves.L.Excellent@ens-lyon.fr</a>></span> wrote:</span>
<blockquote class="gmail_quote" style="margin: 0.0px 0.0px 0.0px 0.8ex;border-left: 1.0px solid rgb(204,204,204);padding-left: 1.0ex;">
<div>
<div class="gmail-m_-3820402716620242094m_-1671753951163694093moz-cite-prefix">
<div class="gmail-m_-3820402716620242094m_-1671753951163694093moz-cite-prefix"><br/>
<span class="gmail-">Thanks for the details of your needs.<br/>
<br/>
For the first application, the sparse RHS feature with distributed<br/>
solution should effectively be fine.</span></div>
</div>
</div>
</blockquote>
<div>I'll add parallel support of this feature in petsc after I'm back the ANL after June 14th.</div>
<div><span class="gmail-HOEnZb"><font color="#888888">Hong</font></span></div>
<div>
<div class="gmail-h5">
<blockquote class="gmail_quote" style="margin: 0.0px 0.0px 0.0px 0.8ex;border-left: 1.0px solid rgb(204,204,204);padding-left: 1.0ex;">
<div>
<div class="gmail-m_-3820402716620242094m_-1671753951163694093moz-cite-prefix">
<div class="gmail-m_-3820402716620242094m_-1671753951163694093moz-cite-prefix"><br/>
For the second one, a future distributed RHS feature (not currently<br/>
available in MUMPS) might help if the centralized sparse RHS is too<br/>
memory consuming, depending on the size of B1.<br/>
<br/>
Regards,<br/>
Jean-Yves, for the MUMPS developers<br/>
<br/>
</div>
<blockquote>
<div style="font-family: Verdana;font-size: 12.0px;">
<div><span> I want to invert a rather large sparse matrix for this using a sparse rhs with centralized input would be ok as long as the solution is distributed. </span></div>
</div>
</blockquote>
</div>
<blockquote>
<div style="font-family: Verdana;font-size: 12.0px;">
<div>and the second application I have in mind is solving a system of the from AX=B where A and B are sparse and B is given by a block matrix of the form B=[B1 0, 0 0] where B1 is dense but the dimension is (much) smaller than that of the whole matrix B.
<div>
<div class="gmail-m_-3820402716620242094h5">
<div style="margin: 10.0px 5.0px 5.0px 10.0px;padding: 10.0px 0.0px 10.0px 10.0px;border-left: 2.0px solid rgb(195,217,229);">
<div style="margin: 0.0px 0.0px 10.0px;"> </div>
<div>
<div>
<div class="gmail_extra">
<div class="gmail_quote">Marius:</div>
<div class="gmail_quote">Current PETSc interface supports sequential sparse <span style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: small;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);float: none;display: inline;">multiple right-hand side, but not distributed.</span></div>
<div class="gmail_quote">It turns out that mumps does not support <span style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: 12.8px;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);float: none;display: inline;">distributed sparse multiple right-hand sides at</span><br style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: 12.8px;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);"/>
<span style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: 12.8px;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);float: none;display: inline;">the moment (see attached email). </span></div>
<div class="gmail_quote"> </div>
<div class="gmail_quote"><span style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: 12.8px;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);float: none;display: inline;"><span style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: 12.8px;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);float: none;display: inline;">Jean-Yves invites you to communicate with him directly. </span></span></div>
<div class="gmail_quote"><span style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: 12.8px;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);float: none;display: inline;"><span style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: 12.8px;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);float: none;display: inline;">Let me know what we can help on this matter, </span></span></div>
<div class="gmail_quote"><span style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: 12.8px;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);float: none;display: inline;"><span style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: 12.8px;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);float: none;display: inline;">e.g., add support for parallel implementation of <span style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: small;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);float: none;display: inline;">sparse </span><span style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: small;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);float: none;display: inline;">multiple right-hand side with </span></span></span></div>
<div class="gmail_quote"><span style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: 12.8px;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);float: none;display: inline;"><span style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: 12.8px;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);float: none;display: inline;"><span style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: small;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);float: none;display: inline;">centralized rhs input?</span></span></span></div>
<div class="gmail_quote"> </div>
<div class="gmail_quote"><span style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: 12.8px;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);float: none;display: inline;"><span style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: 12.8px;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);float: none;display: inline;">Hong</span></span></div>
<div class="gmail_quote"> </div>
<div class="gmail_quote"><span style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: 12.8px;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);float: none;display: inline;">----------------------</span></div>
<div class="gmail_quote">
<div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-gE gmail-m_-3820402716620242094m_-1671753951163694093gmail-iv gmail-m_-3820402716620242094m_-1671753951163694093gmail-gt" style="padding: 10.0px 0.0px 3.0px;font-size: 12.8px;color: rgb(34,34,34);font-family: arial , sans-serif;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);">
<table cellpadding="0" class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-cf gmail-m_-3820402716620242094m_-1671753951163694093gmail-gJ" style="border-collapse: collapse;margin-top: 0.0px;width: auto;">
<tbody>
<tr class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-acZ" style="height: 16.0px;">
<td class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-gF gmail-m_-3820402716620242094m_-1671753951163694093gmail-gK" style="font-family: arial , sans-serif;margin: 0.0px;text-align: left;white-space: nowrap;padding-right: 8.0px;vertical-align: top;width: 270.667px;padding-top: 0.0px;">
<table cellpadding="0" class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-cf gmail-m_-3820402716620242094m_-1671753951163694093gmail-ix" style="border-collapse: collapse;table-layout: fixed;width: 270.667px;">
<tbody>
<tr>
<td style="font-family: arial , sans-serif;margin: 0.0px;">
<h3 class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-iw" style="overflow: hidden;white-space: nowrap;max-width: 92.0%;display: inline-block;font-size: inherit;font-weight: inherit;margin: inherit;"><span style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: 12.8px;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);float: none;display: inline;"><span class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-gD" style="color: rgb(34,34,34);font-size: 12.8px;font-weight: bold;white-space: nowrap;display: inline;vertical-align: top;">Jean-Yves LExcellent</span><span> </span><span class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-go" style="vertical-align: top;color: rgb(85,85,85);"><span><</span><a href="mailto:Jean-Yves.L.Excellent@ens-lyon.fr" onclick="parent.window.location.href='Jean-Yves.L.Excellent@ens-lyon.fr'; return false;" target="_blank">Jean-Yves.L.Excellent@ens-lyon.fr</a><span>></span></span></span></h3>
</td>
</tr>
</tbody>
</table>
</td>
<td class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-gH gmail-m_-3820402716620242094m_-1671753951163694093gmail-bAk" style="font-family: arial , sans-serif;margin: 0.0px;text-align: right;white-space: nowrap;vertical-align: top;color: rgb(34,34,34);">
<div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-gK" style="padding-top: 0.0px;padding-right: 2.0px;"><span style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: 12.8px;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);float: none;display: inline;"><span class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-g3" id="gmail-m_-3820402716620242094m_-1671753951163694093gmail-:pm" style="vertical-align: top;margin-right: 3.0px;" title="Fri, Jun 1, 2018 at 5:14 AM">5:14 AM (3 hours ago)</span></span>
<div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-zd gmail-m_-3820402716620242094m_-1671753951163694093gmail-bi4" style="display: inline-block;height: 20.0px;outline: 0.0px;" title="Not
starred"><span style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: 12.8px;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);float: none;display: inline;"><span class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-KT" style="display: inline-block;height: 19.0px;text-align: center;width: 19.0px;padding: 2.0px;"><img alt="" class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-f gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-KT-JX" src="https://mail.google.com/mail/u/0/images/cleardot.gif" style="margin-top: 0.0px;vertical-align: top;"/></span></span></div>
</div>
</td>
<td class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-gH" style="font-family: arial , sans-serif;margin: 0.0px;text-align: right;white-space: nowrap;vertical-align: top;color: rgb(34,34,34);"> </td>
<td class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-gH gmail-m_-3820402716620242094m_-1671753951163694093gmail-acX gmail-m_-3820402716620242094m_-1671753951163694093gmail-bAm" rowspan="2" style="font-family: arial , sans-serif;margin: 0.0px;text-align: right;white-space: nowrap;vertical-align: top;color: rgb(34,34,34);">
<div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-I gmail-m_-3820402716620242094m_-1671753951163694093gmail-J-J5-Ji gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-I-Js-IF gmail-m_-3820402716620242094m_-1671753951163694093gmail-aaq gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-I-ax7 gmail-m_-3820402716620242094m_-1671753951163694093gmail-L3" style="display: inline-block;font-size: 11.0px;font-weight: bold;text-align: center;white-space: nowrap;margin-right: 0.0px;height: 27.0px;line-height: 27.0px;min-width: 32.0px;outline: 0.0px;padding: 0.0px 8.0px;background-color: rgb(245,245,245);color: rgb(68,68,68);"><span style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: 12.8px;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);float: none;display: inline;"><img alt="" class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-hB gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-I-J3" src="https://mail.google.com/mail/u/0/images/cleardot.gif" style="height: 21.0px;vertical-align: middle;width: 21.0px;"/></span></div>
<div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-I gmail-m_-3820402716620242094m_-1671753951163694093gmail-J-J5-Ji gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-I-Js-Gs gmail-m_-3820402716620242094m_-1671753951163694093gmail-aap gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-I-awG gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-I-ax7 gmail-m_-3820402716620242094m_-1671753951163694093gmail-L3" id="gmail-m_-3820402716620242094m_-1671753951163694093gmail-:px" style="display: inline-block;font-size: 11.0px;font-weight: bold;text-align: center;white-space: nowrap;margin-right: 0.0px;height: 27.0px;line-height: 27.0px;min-width: 21.0px;outline: 0.0px;padding: 0.0px;background-color: rgb(245,245,245);color: rgb(68,68,68);"><span style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: 12.8px;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);float: none;display: inline;"><img alt="" class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-hA gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-I-J3" src="https://mail.google.com/mail/u/0/images/cleardot.gif" style="height: 21.0px;width: 21.0px;vertical-align: middle;"/></span></div>
</td>
</tr>
<tr class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-acZ gmail-m_-3820402716620242094m_-1671753951163694093gmail-xD" style="height: 16.0px;">
<td colspan="3" style="font-family: arial , sans-serif;margin: 0.0px;">
<table cellpadding="0" class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-cf gmail-m_-3820402716620242094m_-1671753951163694093gmail-adz" style="border-collapse: collapse;table-layout: fixed;white-space: nowrap;width: 434.667px;">
<tbody>
<tr>
<td class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-ady" style="font-family: arial , sans-serif;margin: 0.0px;overflow: hidden;white-space: nowrap;">
<div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-iw gmail-m_-3820402716620242094m_-1671753951163694093gmail-ajw" style="overflow: hidden;white-space: nowrap;max-width: 92.0%;display: inline-block;"><span style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: 12.8px;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);float: none;display: inline;"><span class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-hb" style="vertical-align: top;color: rgb(119,119,119);">to<span> </span><span class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-g2">Hong</span>,<span> </span><span class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-g2">mumps-dev</span></span></span></div>
<div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-ajy" style="display: inline-block;margin-left: 5.0px;vertical-align: top;"><span style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: 12.8px;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);float: none;display: inline;"><img alt="" class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-ajz" id="gmail-m_-3820402716620242094m_-1671753951163694093gmail-:pw" src="https://mail.google.com/mail/u/0/images/cleardot.gif" style="padding: 0.0px 0.0px 1.0px;vertical-align: bottom;height: 12.0px;width: 12.0px;"/></span></div>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</div>
<div id="gmail-m_-3820402716620242094m_-1671753951163694093gmail-:pi" style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: medium;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);">
<div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-qQVYZb"> </div>
<div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-utdU2e"> </div>
<div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-btm"> </div>
</div>
<div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-" style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: medium;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);">
<div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-aHl"> </div>
<div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-ajV" id="gmail-m_-3820402716620242094m_-1671753951163694093gmail-:pv" style="outline: none;padding: 5.0px 0.0px;width: 22.0px;margin: 0.0px;">
<div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-ajR" id="gmail-m_-3820402716620242094m_-1671753951163694093gmail-:qe" style="background-color: rgb(241,241,241);border: 1.0px solid rgb(221,221,221);clear: both;line-height: 6.0px;outline: none;width: 20.0px;"><span style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: 12.8px;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);float: none;display: inline;"><img class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-ajT" src="https://ssl.gstatic.com/ui/v1/icons/mail/images/cleardot.gif" style="height: 8.0px;width: 20.0px;"/></span></div>
</div>
<div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-ii gmail-m_-3820402716620242094m_-1671753951163694093gmail-gt gmail-m_-3820402716620242094m_-1671753951163694093gmail-adP" id="gmail-m_-3820402716620242094m_-1671753951163694093gmail-:pk" style="font-size: 12.8px;direction: ltr;margin: 5.0px 15.0px 0.0px 0.0px;padding-bottom: 5.0px;">
<div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-a3s gmail-m_-3820402716620242094m_-1671753951163694093gmail-aXjCH gmail-m_-3820402716620242094m_-1671753951163694093gmail-m163bad8b399eea70" id="gmail-m_-3820402716620242094m_-1671753951163694093gmail-:pj" style="overflow: hidden;"><span style="color: rgb(34,34,34);font-family: arial , sans-serif;font-size: 12.8px;font-style: normal;font-weight: 400;letter-spacing: normal;text-indent: 0.0px;text-transform: none;white-space: normal;word-spacing: 0.0px;background-color: rgb(255,255,255);float: none;display: inline;">Hello,<br/>
<br/>
We do not support distributed sparse multiple right-hand sides at<br/>
the moment. From the feedback we have from applications, the<br/>
right-hand sides are often very sparse, and having them distributed<br/>
did not seem critical.<br/>
<br/>
Since we are specifying a distributed right-hand sides feature at the<br/>
moment, could you let us know in more detail the need regarding<br/>
distributed sparse right-hand side (e.g., do all columns have the same<br/>
nonzero structure in that case) or put us in contact with the user who<br/>
needs this?<br/>
<br/>
Thanks,<br/>
Jean-Yves and Patrick</span></div>
</div>
</div>
</div>
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin: 0.0px 0.0px 0.0px 0.8ex;border-left: 1.0px solid rgb(204,204,204);padding-left: 1.0ex;">Thanks a lot guys, very helpful.<br/>
<span> <br/>
<br/>
<br/>
<br/>
I see MUMPS <a href="http://mumps.enseeiht.fr/" target="_blank">http://mumps.enseeiht.fr/</a></span><br/>
<br/>
Sparse multiple right-hand side, distributed solution; Exploitation of sparsity in the right-hand sidesPETSc interface computes mumps distributed solution as default (this is not new) (ICNTL(21) = 1)<br/>
<span> <br/>
I will add support for Sparse multiple right-hand side.<br/>
<br/>
Hong<br/>
</span><br/>
<span>On Thu, May 31, 2018 at 11:25 AM, Smith, Barry F. <<a href="mailto:bsmith@mcs.anl.gov" onclick="parent.window.location.href='bsmith@mcs.anl.gov'; return false;" target="_blank">bsmith@mcs.anl.gov</a>[mailto:<a href="mailto:bsmith@mcs.anl.gov" onclick="parent.window.location.href='bsmith@mcs.anl.gov'; return false;" target="_blank">bsmith@mcs.anl.gov</a>]> wrote:<br/>
Hong,<br/>
<br/>
Can you see about adding support for distributed right hand side?<br/>
<br/>
Thanks<br/>
<br/>
Barry</span><br/>
<br/>
<span>> On May 31, 2018, at 2:37 AM, Marius Buerkle <<a href="mailto:mbuerkle@web.de" onclick="parent.window.location.href='mbuerkle@web.de'; return false;" target="_blank">mbuerkle@web.de</a>[mailto:<a href="mailto:mbuerkle@web.de" onclick="parent.window.location.href='mbuerkle@web.de'; return false;" target="_blank">mbuerkle@web.de</a>]> wrote:<br/>
><br/>
> The fix for MAT_NEW_NONZERO_LOCATIONS, thanks again.<br/>
><br/>
> I have yet another question, sorry. The recent version of MUMPS supports distributed and sparse RHS is there any chance that this will be supported in PETSc in the near future?<br/>
><br/>
> <br/>
> <br/>
></span><br/>
<span>>> On May 30, 2018, at 6:55 PM, Marius Buerkle <<a href="mailto:mbuerkle@web.de" onclick="parent.window.location.href='mbuerkle@web.de'; return false;" target="_blank">mbuerkle@web.de</a>[mailto:<a href="mailto:mbuerkle@web.de" onclick="parent.window.location.href='mbuerkle@web.de'; return false;" target="_blank">mbuerkle@web.de</a>]> wrote:<br/>
>><br/>
>> Thanks for the quick fix, I will test it and report back.<br/>
>> I have another maybe related question, if MAT_NEW_NONZERO_LOCATIONS is true and let's say 1 new nonzero position is created it does not allocated 1 but several new nonzeros but only use 1.<br/>
><br/>
> Correct<br/>
><br/>
>> I think that is normal, right?<br/>
><br/>
> Yes<br/>
><br/>
>> But, at least as far as I understand the manual, a subsequent call of mat assemble with<br/>
>> MAT_FINAL_ASSEMBLY should compress out the unused allocations and release the memory, is this correct?<br/>
><br/>
> It "compresses it out" (by shifting all the nonzero entries to the beginning of the internal i, j, and a arrays), but does NOT release any memory. Since the values are stored in one big contiguous array (obtained with a single malloc) it cannot just free part of the array, so the extra locations just sit harmlessly at the end if the array unused.<br/>
><br/>
>> If so, this did not work for me, even after doing<br/>
>> MAT_FINAL_ASSEMBLY the unused nonzero allocations remain. Is this normal?<br/>
><br/>
> Yes,<br/>
><br/>
> Barry<br/>
><br/>
>><br/>
>>><br/>
>>> Fixed in the branch barry/fix-mat-new-nonzero-locations/maint<br/>
>>><br/>
>>> Once this passes testing it will go into the maint branch and then the next patch release but you can use it now in the branch barry/fix-mat-new-nonzero-locations/maint<br/>
>>><br/>
>>> Thanks for the report and reproducible example<br/>
>>><br/>
>>> Barry<br/>
>>><br/>
>>></span>
<div>
<div class="gmail-m_-3820402716620242094m_-1671753951163694093h5">>>>> On May 29, 2018, at 7:51 PM, Marius Buerkle <<a href="mailto:mbuerkle@web.de" onclick="parent.window.location.href='mbuerkle@web.de'; return false;" target="_blank">mbuerkle@web.de</a>[mailto:<a href="mailto:mbuerkle@web.de" onclick="parent.window.location.href='mbuerkle@web.de'; return false;" target="_blank">mbuerkle@web.de</a>]> wrote:<br/>
>>>><br/>
>>>> Sure, I made a small reproducer, it is Fortran though I hope that is ok. If MAT_NEW_NONZERO_LOCATIONS is set to false I get an error, if it is set to true the new nonzero element is inserted, if MAT_NEW_NONZERO_LOCATIONS is false and either MAT_NEW_NONZERO_LOCATION_ERR or MAT_NEW_NONZERO_ALLOCATION_ERR is set to false afterwards then the new nonzero is also created without an error, but if MAT_NEW_NONZERO_LOCATIONS is set to false after MAT_NEW_NONZERO_LOCATION_ERR/MAT_NEW_NONZERO_ALLOCATION_ERR have been set to false I get an error again.<br/>
>>>><br/>
>>>><br/>
>>>> program newnonzero<br/>
>>>> #include <petsc/finclude/petscmat.h><br/>
>>>> use petscmat<br/>
>>>> implicit none<br/>
>>>><br/>
>>>> Mat :: A<br/>
>>>> PetscInt :: dnnz,onnz,n,m,idxm(1),idxn(1),nl1,nl2<br/>
>>>> PetscScalar :: v(1)<br/>
>>>> PetscReal :: info(MAT_INFO_SIZE)<br/>
>>>> PetscErrorCode :: ierr<br/>
>>>><br/>
>>>> integer :: nproc,iproc,i<br/>
>>>><br/>
>>>> call PetscInitialize(PETSC_NULL_CHARACTER,ierr)<br/>
>>>><br/>
>>>> call MPI_COMM_SIZE(PETSC_COMM_WORLD, nproc,ierr)<br/>
>>>><br/>
>>>> call MPI_Comm_rank( PETSC_COMM_WORLD, iproc, ierr )<br/>
>>>><br/>
>>>> n=3<br/>
>>>> m=n<br/>
>>>> call MatCreateAIJ(PETSC_COMM_WORLD,PETSC_DECIDE,PETSC_DECIDE,n,m,1,PETSC_NULL_INTEGER,0,PETSC_NULL_INTEGER,A,ierr)<br/>
>>>><br/>
>>>><br/>
>>>> call MatGetOwnershipRange(A,nl1,nl2,ierr)<br/>
>>>> do i=nl1,nl2-1<br/>
>>>> idxn(1)=i<br/>
>>>> idxm(1)=i<br/>
>>>> v(1)=1d0<br/>
>>>> call MatSetValues(A,1,idxn,1,idxm, v,INSERT_VALUES,ierr)<br/>
>>>> end do<br/>
>>>> call MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY,ierr)<br/>
>>>> call MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY,ierr)<br/>
>>>><br/>
>>>> call MatSetOption(A,MAT_NEW_NONZERO_LOCATIONS,PETSC_FALSE,ierr)<br/>
>>>> !~ call MatSetOption(A,MAT_NEW_NONZERO_LOCATION_ERR,PETSC_FALSE,ierr)<br/>
>>>> !~ call MatSetOption(A,MAT_NEW_NONZERO_ALLOCATION_ERR ,PETSC_FALSE,ierr)<br/>
>>>> !~ call MatSetOption(A,MAT_NEW_NONZERO_LOCATIONS,PETSC_FALSE,ierr)<br/>
>>>><br/>
>>>><br/>
>>>> idxn(1)=0<br/>
>>>> idxm(1)=n-1<br/>
>>>> if ((idxn(1).ge.nl1).and.(idxn(1).le.nl2-1)) then<br/>
>>>> v(1)=2d0<br/>
>>>> call MatSetValues(A,1,idxn,1,idxm, v,INSERT_VALUES,ierr)<br/>
>>>> end if<br/>
>>>> call MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY,ierr)<br/>
>>>> call MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY,ierr)<br/>
>>>><br/>
>>>> if ((idxn(1).ge.nl1).and.(idxn(1).le.nl2-1)) then<br/>
>>>> v(1)=2d0<br/>
>>>> call MatGetValues(A,1,idxn,1,idxm, v,ierr)<br/>
>>>> write(6,*) v<br/>
>>>> end if<br/>
>>>><br/>
>>>> call PetscFinalize(ierr)<br/>
>>>><br/>
>>>> end program newnonzero<br/>
>>>><br/>
>>>><br/>
>>>><br/>
>>>> $ mpiexec.hydra -n 3 ./a.out<br/>
>>>> [0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br/>
>>>> [0]PETSC ERROR: Argument out of range<br/>
>>>> [0]PETSC ERROR: Inserting a new nonzero at global row/column (0, 2) into matrix</div>
</div>
>>>> [0]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html%5Bhttp://www.mcs.anl.gov/petsc/documentation/faq.html%5D" target="_blank">http://www.mcs.anl.gov/petsc/documentation/faq.html[http://www.mcs.anl.gov/petsc/documentation/faq.html]</a> for trouble shooting.<br/>
<span class="gmail-m_-3820402716620242094m_-1671753951163694093im gmail-m_-3820402716620242094m_-1671753951163694093HOEnZb">>>>> [0]PETSC ERROR: Petsc Release Version 3.9.2, May, 20, 2018<br/>
>>>> [0]PETSC ERROR: ./a.out on a named tono-hpc1 by marius Wed May 30 09:42:40 2018<br/>
>>>> [0]PETSC ERROR: Configure options --prefix=/home/marius/prog/petsc/3.9.2 --download-elemental=yes --download-metis=yes --download-parmetis=yes --download-mumps=yes --with-scalapack-lib="/home/marius/intel/compilers_and_libraries_2018.2.199/linux/mkl/lib/intel64/libmkl_scalapack_lp64.a -Wl,--start-group /home/marius/intel/compilers_and_libraries_2018.2.199/linux/mkl/lib/intel64/libmkl_intel_lp64.a /home/marius/intel/compilers_and_libraries_2018.2.199/linux/mkl/lib/intel64/libmkl_sequential.a /home/marius/intel/compilers_and_libraries_2018.2.199/linux/mkl/lib/intel64/libmkl_core.a /home/marius/intel/compilers_and_libraries_2018.2.199/linux/mkl/lib/intel64/libmkl_blacs_intelmpi_lp64.a -Wl,--end-group -lpthread -lm -ldl" --FC=mpiifort --CC=mpicc --CXX=mpicxx --with-scalar-type=complex --with-mpi-dir= --with-blaslapack-lib="/home/marius/intel/compilers_and_libraries_2018.2.199/linux/mkl/lib/intel64/libmkl_scalapack_lp64.a -Wl,--start-group /home/marius/intel/compilers_and_libraries_2018.2.199/linux/mkl/lib/intel64/libmkl_intel_lp64.a /home/marius/intel/compilers_and_libraries_2018.2.199/linux/mkl/lib/intel64/libmkl_sequential.a /home/marius/intel/compilers_and_libraries_2018.2.199/linux/mkl/lib/intel64/libmkl_core.a /home/marius/intel/compilers_and_libraries_2018.2.199/linux/mkl/lib/intel64/libmkl_blacs_intelmpi_lp64.a -Wl,--end-group -lpthread -lm -ldl" --with-cxx-dialect=C++11 --download-superlu_dist=yes --download-ptscotch=yes --with-x --with-debugging=1 --download-superlu=yes --with-mkl_cpardiso=1 --with-mkl_pardiso=1 --with-scalapack=1<br/>
>>>> [0]PETSC ERROR: #1 MatSetValues_MPIAIJ() line 607 in /home/marius/prog/petsc/petsc-3.9.2/src/mat/impls/aij/mpi/mpiaij.c<br/>
>>>> [0]PETSC ERROR: #2 MatSetValues() line 1312 in /home/marius/prog/petsc/petsc-3.9.2/src/mat/interface/matrix.c<br/>
>>>> (0.000000000000000E+000,0.000000000000000E+000)<br/>
>>>><br/>
>>>><br/>
>>>><br/>
>>>> Please send complete error message; type of matrix used etc. Ideally code that demonstrates the problem.<br/>
>>>><br/>
>>>> Barry<br/>
>>>><br/>
>>>></span>
<div class="gmail-m_-3820402716620242094m_-1671753951163694093HOEnZb">
<div class="gmail-m_-3820402716620242094m_-1671753951163694093h5">>>>>> On May 29, 2018, at 3:31 AM, Marius Buerkle <<a href="mailto:mbuerkle@web.de" onclick="parent.window.location.href='mbuerkle@web.de'; return false;" target="_blank">mbuerkle@web.de</a>[mailto:<a href="mailto:mbuerkle@web.de" onclick="parent.window.location.href='mbuerkle@web.de'; return false;" target="_blank">mbuerkle@web.de</a>]> wrote:<br/>
>>>>><br/>
>>>>><br/>
>>>>> Hi,<br/>
>>>>><br/>
>>>>> I tried to set MAT_NEW_NONZERO_LOCATIONS to false, as far as I understood MatSetValues should simply ignore entries which would give rise to new nonzero values not creating a new entry and not cause an error, but I get "[1]PETSC ERROR: Inserting a new nonzero at global row/column". Is this option supposed to work or not?<br/>
>>>><br/>
>>><br/>
>>><br/>
> <br/>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<p> </p>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</div>
</div></div></body></html>