<div dir="ltr"><span style="font-size:12.8px;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">Marius:</span><div><span style="font-size:12.8px">I added support for parallel sparse RHS (in host) for MatMatSolve_MUMPS()</span></div><div><span style="font-size:12.8px"><a href="https://bitbucket.org/petsc/petsc/commits/2b691707dd0cf456c808def006e14b6f56b364b6">https://bitbucket.org/petsc/petsc/commits/2b691707dd0cf456c808def006e14b6f56b364b6</a></span></div><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px">It is in the branch hzhang/mumps-spRHS.</span></div><div><span style="font-size:12.8px">You may test it.</span></div><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px">I'll further cleanup the routine, test it, then merge it to petsc.</span></div><div><span style="font-size:12.8px">Hong<br></span><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jun 4, 2018 at 9:54 AM, Hong <span dir="ltr"><<a href="mailto:hzhang@mcs.anl.gov" target="_blank">hzhang@mcs.anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span class="gmail-">On Mon, Jun 4, 2018 at 1:03 PM, Jean-Yves LExcellent <span dir="ltr"><<a href="mailto:Jean-Yves.L.Excellent@ens-lyon.fr" target="_blank">Jean-Yves.L.Excellent@ens-<wbr>lyon.fr</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
  
    
  
  <div bgcolor="#FFFFFF">
    <div class="gmail-m_-3820402716620242094m_-1671753951163694093moz-cite-prefix">
      <div class="gmail-m_-3820402716620242094m_-1671753951163694093moz-cite-prefix"><br>
        Thanks for the details of your needs.<br>
        <br>
        For the first application, the sparse RHS feature with
        distributed<br>
        solution should effectively be fine.<br></div></div></div></blockquote></span><div>I'll add parallel support of this feature in petsc after I'm back the ANL after June 14th.</div><span class="gmail-HOEnZb"><font color="#888888"><div>Hong</div></font></span><div><div class="gmail-h5"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="#FFFFFF"><div class="gmail-m_-3820402716620242094m_-1671753951163694093moz-cite-prefix"><div class="gmail-m_-3820402716620242094m_-1671753951163694093moz-cite-prefix">
        <br>
        For the second one, a future distributed RHS feature (not
        currently<br>
        available in MUMPS) might help if the centralized sparse RHS is
        too<br>
        memory consuming, depending on the size of B1.<br>
        <br>
        Regards,<br>
        Jean-Yves, for the MUMPS developers<br>
        <br>
        <br>
      </div><span>
      <blockquote type="cite">
        <div style="font-family:Verdana;font-size:12px">
          <div> I want to  invert a rather large sparse matrix for this
            using a sparse rhs with centralized input would be ok as
            long as the solution is distributed.  </div>
        </div>
      </blockquote>
    </span></div>
    <blockquote type="cite">
      <div style="font-family:Verdana;font-size:12px">
        <div>and the second application I have in mind is solving a
          system of the from AX=B where A and B are sparse and B is
          given by a block matrix of the form B=[B1 0, 0 0] where B1 is
          dense but the dimension is (much) smaller than that of the
          whole matrix B.
          <div><div class="gmail-m_-3820402716620242094h5"><div name="quote" style="margin:10px 5px 5px 10px;padding:10px 0px 10px 10px;border-left:2px solid rgb(195,217,229);word-wrap:break-word">
            <div style="margin:0px 0px 10px"> </div>
            <div name="quoted-content">
              <div>
                <div class="gmail_extra">
                  <div class="gmail_quote">Marius:</div>
                  <div class="gmail_quote">Current PETSc interface
                    supports sequential sparse <span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline">multiple right-hand side,
                      but not distributed.</span></div>
                  <div class="gmail_quote">It turns out that mumps does
                    not support <span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline">distributed sparse multiple
                      right-hand sides at</span><br style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
                    <span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline">the moment (see attached
                      email). </span></div>
                  <div class="gmail_quote"> </div>
                  <div class="gmail_quote"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline">Jean-Yves invites you to
                        communicate with him directly. </span></span></div>
                  <div class="gmail_quote"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline">Let me know what we can
                        help on this matter, </span></span></div>
                  <div class="gmail_quote"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline">e.g., add support for
                        parallel implementation of <span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline">sparse </span><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline">multiple
                          right-hand side with </span></span></span></div>
                  <div class="gmail_quote"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline">centralized
                          rhs input?</span></span></span></div>
                  <div class="gmail_quote"> </div>
                  <div class="gmail_quote"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline">Hong</span></span></div>
                  <div class="gmail_quote"> </div>
                  <div class="gmail_quote"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline">----------------------</span></div>
                  <div class="gmail_quote">
                    <div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-gE gmail-m_-3820402716620242094m_-1671753951163694093gmail-iv gmail-m_-3820402716620242094m_-1671753951163694093gmail-gt" style="padding:10px 0px 3px;font-size:12.8px;color:rgb(34,34,34);font-family:arial,sans-serif;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
                      <table class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-cf gmail-m_-3820402716620242094m_-1671753951163694093gmail-gJ" style="border-collapse:collapse;margin-top:0px;width:auto" cellpadding="0">
                        <tbody>
                          <tr class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-acZ" style="height:16px">
                            <td class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-gF gmail-m_-3820402716620242094m_-1671753951163694093gmail-gK" style="font-family:arial,sans-serif;margin:0px;text-align:left;white-space:nowrap;padding-right:8px;vertical-align:top;width:270.667px;padding-top:0px">
                              <table class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-cf gmail-m_-3820402716620242094m_-1671753951163694093gmail-ix" style="border-collapse:collapse;table-layout:fixed;width:270.667px" cellpadding="0">
                                <tbody>
                                  <tr>
                                    <td style="font-family:arial,sans-serif;margin:0px">
                                      <h3 class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-iw" style="overflow:hidden;white-space:nowrap;max-width:92%;display:inline-block;font-size:inherit;font-weight:inherit;margin:inherit"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><span class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-gD" style="color:rgb(34,34,34);font-size:12.8px;font-weight:bold;white-space:nowrap;display:inline;vertical-align:top">Jean-Yves
                                            LExcellent</span><span> </span><span class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-go" style="vertical-align:top;color:rgb(85,85,85)"><span><</span><a href="mailto:Jean-Yves.L.Excellent@ens-lyon.fr" target="_blank">Jean-Yves.L.Excell<wbr>ent@ens-lyon.fr</a><span>></span></span></span></h3>
                                    </td>
                                  </tr>
                                </tbody>
                              </table>
                            </td>
                            <td class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-gH gmail-m_-3820402716620242094m_-1671753951163694093gmail-bAk" style="font-family:arial,sans-serif;margin:0px;text-align:right;white-space:nowrap;vertical-align:top;color:rgb(34,34,34)">
                              <div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-gK" style="padding-top:0px;padding-right:2px"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><span class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-g3" id="gmail-m_-3820402716620242094m_-1671753951163694093gmail-:pm" style="vertical-align:top;margin-right:3px" title="Fri, Jun 1, 2018 at 5:14 AM">5:14
                                    AM (3 hours ago)</span></span>
                                <div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-zd gmail-m_-3820402716620242094m_-1671753951163694093gmail-bi4" style="display:inline-block;height:20px;outline:0px" title="Not
                                  starred"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><span class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-KT" style="display:inline-block;height:19px;text-align:center;width:19px;padding:2px"><img alt="" class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-f gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-KT-JX" src="https://mail.google.com/mail/u/0/images/cleardot.gif" style="margin-top: 0px; vertical-align: top;"></span></span></div>
                              </div>
                            </td>
                            <td class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-gH" style="font-family:arial,sans-serif;margin:0px;text-align:right;white-space:nowrap;vertical-align:top;color:rgb(34,34,34)"> </td>
                            <td class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-gH gmail-m_-3820402716620242094m_-1671753951163694093gmail-acX gmail-m_-3820402716620242094m_-1671753951163694093gmail-bAm" rowspan="2" style="font-family:arial,sans-serif;margin:0px;text-align:right;white-space:nowrap;vertical-align:top;color:rgb(34,34,34)">
                              <div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-I gmail-m_-3820402716620242094m_-1671753951163694093gmail-J-J5-Ji gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-I-Js-IF gmail-m_-3820402716620242094m_-1671753951163694093gmail-aaq gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-I-ax7 gmail-m_-3820402716620242094m_-1671753951163694093gmail-L3" style="display:inline-block;font-size:11px;font-weight:bold;text-align:center;white-space:nowrap;margin-right:0px;height:27px;line-height:27px;min-width:32px;outline:0px;padding:0px 8px;background-color:rgb(245,245,245);color:rgb(68,68,68)"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><img alt="" class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-hB gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-I-J3" src="https://mail.google.com/mail/u/0/images/cleardot.gif" style="height: 21px; vertical-align: middle; width: 21px;"></span></div>
                              <div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-I gmail-m_-3820402716620242094m_-1671753951163694093gmail-J-J5-Ji gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-I-Js-Gs gmail-m_-3820402716620242094m_-1671753951163694093gmail-aap gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-I-awG gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-I-ax7 gmail-m_-3820402716620242094m_-1671753951163694093gmail-L3" id="gmail-m_-3820402716620242094m_-1671753951163694093gmail-:px" style="display:inline-block;font-size:11px;font-weight:bold;text-align:center;white-space:nowrap;margin-right:0px;height:27px;line-height:27px;min-width:21px;outline:0px;padding:0px;background-color:rgb(245,245,245);color:rgb(68,68,68)"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><img alt="" class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-hA gmail-m_-3820402716620242094m_-1671753951163694093gmail-T-I-J3" src="https://mail.google.com/mail/u/0/images/cleardot.gif" style="height: 21px; width: 21px; vertical-align: middle;"></span></div>
                            </td>
                          </tr>
                          <tr class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-acZ gmail-m_-3820402716620242094m_-1671753951163694093gmail-xD" style="height:16px">
                            <td colspan="3" style="font-family:arial,sans-serif;margin:0px">
                              <table class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-cf gmail-m_-3820402716620242094m_-1671753951163694093gmail-adz" style="border-collapse:collapse;table-layout:fixed;white-space:nowrap;width:434.667px" cellpadding="0">
                                <tbody>
                                  <tr>
                                    <td class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-ady" style="font-family:arial,sans-serif;margin:0px;overflow:hidden;white-space:nowrap">
                                      <div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-iw gmail-m_-3820402716620242094m_-1671753951163694093gmail-ajw" style="overflow:hidden;white-space:nowrap;max-width:92%;display:inline-block"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><span class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-hb" style="vertical-align:top;color:rgb(119,119,119)">to<span> </span><span class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-g2">Hong</span>,<span> </span><span class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-g2">mumps-dev</span></span></span></div>
                                      <div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-ajy" style="display:inline-block;margin-left:5px;vertical-align:top"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><img alt="" class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-ajz" id="gmail-m_-3820402716620242094m_-1671753951163694093gmail-:pw" src="https://mail.google.com/mail/u/0/images/cleardot.gif" style="background: url("") -60px -100px no-repeat; padding: 0px 0px 1px; vertical-align: bottom; height: 12px; width: 12px;"></span></div>
                                    </td>
                                  </tr>
                                </tbody>
                              </table>
                            </td>
                          </tr>
                        </tbody>
                      </table>
                    </div>
                    <div id="gmail-m_-3820402716620242094m_-1671753951163694093gmail-:pi" style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:medium;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
                      <div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-qQVYZb"> </div>
                      <div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-utdU2e"> </div>
                      <div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-btm"> </div>
                    </div>
                    <div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-" style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:medium;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
                      <div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-aHl"> </div>
                      <div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-ajV" id="gmail-m_-3820402716620242094m_-1671753951163694093gmail-:pv" style="outline:none;padding:5px 0px;width:22px;margin:0px">
                        <div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-ajR" id="gmail-m_-3820402716620242094m_-1671753951163694093gmail-:qe" style="background-color:rgb(241,241,241);border:1px solid rgb(221,221,221);clear:both;line-height:6px;outline:none;width:20px"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline"><img class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-ajT" src="https://ssl.gstatic.com/ui/v1/icons/mail/images/cleardot.gif" style="background: url("") no-repeat; height: 8px; width: 20px;"></span></div>
                      </div>
                      <div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-ii gmail-m_-3820402716620242094m_-1671753951163694093gmail-gt gmail-m_-3820402716620242094m_-1671753951163694093gmail-adP" id="gmail-m_-3820402716620242094m_-1671753951163694093gmail-:pk" style="font-size:12.8px;direction:ltr;margin:5px 15px 0px 0px;padding-bottom:5px">
                        <div class="gmail-m_-3820402716620242094m_-1671753951163694093gmail-a3s gmail-m_-3820402716620242094m_-1671753951163694093gmail-aXjCH gmail-m_-3820402716620242094m_-1671753951163694093gmail-m163bad8b399eea70" id="gmail-m_-3820402716620242094m_-1671753951163694093gmail-:pj" style="overflow:hidden"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);float:none;display:inline">Hello,<br>
                            <br>
                            We do not support distributed sparse
                            multiple right-hand sides at<br>
                            the moment. From the feedback we have from
                            applications, the<br>
                            right-hand sides are often very sparse, and
                            having them distributed<br>
                            did not seem critical.<br>
                            <br>
                            Since we are specifying a distributed
                            right-hand sides feature at the<br>
                            moment, could you let us know in more detail
                            the need regarding<br>
                            distributed sparse right-hand side (e.g., do
                            all columns have the same<br>
                            nonzero structure in that case) or put us in
                            contact with the user who<br>
                            needs this?<br>
                            <br>
                            Thanks,<br>
                            Jean-Yves and Patrick</span></div>
                      </div>
                    </div>
                  </div>
                  <div class="gmail_quote"> 
                    <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Thanks a lot guys,
                      very helpful.<br>
                      <span> <br>
                        <br>
                        <br>
                        <br>
                        I see MUMPS <a href="http://mumps.enseeiht.fr/" target="_blank">http://mumps.enseeiht.fr<wbr>/</a></span><br>
                      <br>
                      Sparse multiple right-hand side, distributed
                      solution; Exploitation of sparsity in the
                      right-hand sidesPETSc interface computes
                      mumps distributed solution as default (this is not
                      new) (ICNTL(21) = 1)<br>
                      <span> <br>
                        I will add support for Sparse multiple
                        right-hand side.<br>
                         <br>
                        Hong<br>
                         </span><br>
                      <span>On Thu, May 31, 2018 at 11:25 AM, Smith,
                        Barry F. <<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>[mailto:<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsm<wbr>ith@mcs.anl.gov</a>]>
                        wrote:<br>
                          Hong,<br>
                        <br>
                            Can you see about adding support for
                        distributed right hand side?<br>
                        <br>
                            Thanks<br>
                        <br>
                              Barry</span><br>
                      <br>
                      <span>> On May 31, 2018, at 2:37 AM, Marius
                        Buerkle <<a href="mailto:mbuerkle@web.de" target="_blank">mbuerkle@web.de</a>[mailto:<a href="mailto:mbuerkle@web.de" target="_blank">mbuerk<wbr>le@web.de</a>]>
                        wrote:<br>
                        ><br>
                        > The fix for MAT_NEW_NONZERO_LOCATIONS,
                        thanks again.<br>
                        ><br>
                        > I have yet another question, sorry. The
                        recent version of MUMPS supports distributed and
                        sparse RHS is there any chance that this will be
                        supported in PETSc in the near future?<br>
                        ><br>
                        > <br>
                        > <br>
                        ></span><br>
                      <span>>> On May 30, 2018, at 6:55 PM, Marius
                        Buerkle <<a href="mailto:mbuerkle@web.de" target="_blank">mbuerkle@web.de</a>[mailto:<a href="mailto:mbuerkle@web.de" target="_blank">mbuerk<wbr>le@web.de</a>]>
                        wrote:<br>
                        >><br>
                        >> Thanks for the quick fix, I will test
                        it and report back.<br>
                        >> I have another maybe related question,
                        if MAT_NEW_NONZERO_LOCATIONS is true and let's
                        say 1 new nonzero position is created it does
                        not allocated 1 but several new nonzeros but
                        only use 1.<br>
                        ><br>
                        > Correct<br>
                        ><br>
                        >> I think that is normal, right?<br>
                        ><br>
                        > Yes<br>
                        ><br>
                        >> But, at least as far as I understand
                        the manual, a subsequent call of mat assemble
                        with<br>
                        >> MAT_FINAL_ASSEMBLY should compress out
                        the unused allocations and release the memory,
                        is this correct?<br>
                        ><br>
                        > It "compresses it out" (by shifting all the
                        nonzero entries to the beginning of the internal
                        i, j, and a arrays), but does NOT release any
                        memory. Since the values are stored in one big
                        contiguous array (obtained with a single malloc)
                        it cannot just free part of the array, so the
                        extra locations just sit harmlessly at the end
                        if the array unused.<br>
                        ><br>
                        >> If so, this did not work for me, even
                        after doing<br>
                        >> MAT_FINAL_ASSEMBLY the unused nonzero
                        allocations remain. Is this normal?<br>
                        ><br>
                        > Yes,<br>
                        ><br>
                        > Barry<br>
                        ><br>
                        >><br>
                        >>><br>
                        >>> Fixed in the branch
                        barry/fix-mat-new-nonzero-loca<wbr>tions/maint<br>
                        >>><br>
                        >>> Once this passes testing it will go
                        into the maint branch and then the next patch
                        release but you can use it now in the branch
                        barry/fix-mat-new-nonzero-loca<wbr>tions/maint<br>
                        >>><br>
                        >>> Thanks for the report and
                        reproducible example<br>
                        >>><br>
                        >>> Barry<br>
                        >>><br>
                        >>></span>
                      <div>
                        <div class="gmail-m_-3820402716620242094m_-1671753951163694093h5">>>>> On May 29,
                          2018, at 7:51 PM, Marius Buerkle <<a href="mailto:mbuerkle@web.de" target="_blank">mbuerkle@web.de</a>[mailto:<a href="mailto:mbuerkle@web.de" target="_blank">mbuerk<wbr>le@web.de</a>]>
                          wrote:<br>
                          >>>><br>
                          >>>> Sure, I made a small
                          reproducer, it is Fortran though I hope that
                          is ok. If MAT_NEW_NONZERO_LOCATIONS is set to
                          false I get an error, if it is set to true the
                          new nonzero element is inserted, if
                          MAT_NEW_NONZERO_LOCATIONS is false and either
                          MAT_NEW_NONZERO_LOCATION_ERR or
                          MAT_NEW_NONZERO_ALLOCATION_ERR is set to false
                          afterwards then the new nonzero is also
                          created without an error, but if
                          MAT_NEW_NONZERO_LOCATIONS is set to false
                          after
                          MAT_NEW_NONZERO_LOCATION_ERR/M<wbr>AT_NEW_NONZERO_ALLOCATION_ERR
                          have been set to false I get an error again.<br>
                          >>>><br>
                          >>>><br>
                          >>>> program newnonzero<br>
                          >>>> #include
                          <petsc/finclude/petscmat.h><br>
                          >>>> use petscmat<br>
                          >>>> implicit none<br>
                          >>>><br>
                          >>>> Mat :: A<br>
                          >>>> PetscInt ::
                          dnnz,onnz,n,m,idxm(1),idxn(1),<wbr>nl1,nl2<br>
                          >>>> PetscScalar :: v(1)<br>
                          >>>> PetscReal ::
                          info(MAT_INFO_SIZE)<br>
                          >>>> PetscErrorCode :: ierr<br>
                          >>>><br>
                          >>>> integer :: nproc,iproc,i<br>
                          >>>><br>
                          >>>> call
                          PetscInitialize(PETSC_NULL_CHA<wbr>RACTER,ierr)<br>
                          >>>><br>
                          >>>> call
                          MPI_COMM_SIZE(PETSC_COMM_WORLD<wbr>, nproc,ierr)<br>
                          >>>><br>
                          >>>> call MPI_Comm_rank(
                          PETSC_COMM_WORLD, iproc, ierr )<br>
                          >>>><br>
                          >>>> n=3<br>
                          >>>> m=n<br>
                          >>>> call
MatCreateAIJ(PETSC_COMM_WORLD,<wbr>PETSC_DECIDE,PETSC_DECIDE,n,m,<wbr>1,PETSC_NULL_INTEGER,0,PETSC_N<wbr>ULL_INTEGER,A,ierr)<br>
                          >>>><br>
                          >>>><br>
                          >>>> call
                          MatGetOwnershipRange(A,nl1,nl2<wbr>,ierr)<br>
                          >>>> do i=nl1,nl2-1<br>
                          >>>> idxn(1)=i<br>
                          >>>> idxm(1)=i<br>
                          >>>> v(1)=1d0<br>
                          >>>> call
                          MatSetValues(A,1,idxn,1,idxm,
                          v,INSERT_VALUES,ierr)<br>
                          >>>> end do<br>
                          >>>> call
                          MatAssemblyBegin(A,MAT_FINAL_A<wbr>SSEMBLY,ierr)<br>
                          >>>> call
                          MatAssemblyEnd(A,MAT_FINAL_ASS<wbr>EMBLY,ierr)<br>
                          >>>><br>
                          >>>> call
                          MatSetOption(A,MAT_NEW_NONZERO<wbr>_LOCATIONS,PETSC_FALSE,ierr)<br>
                          >>>> !~ call
                          MatSetOption(A,MAT_NEW_NONZERO<wbr>_LOCATION_ERR,PETSC_FALSE,<wbr>ierr)<br>
                          >>>> !~ call
                          MatSetOption(A,MAT_NEW_NONZERO<wbr>_ALLOCATION_ERR
                          ,PETSC_FALSE,ierr)<br>
                          >>>> !~ call
                          MatSetOption(A,MAT_NEW_NONZERO<wbr>_LOCATIONS,PETSC_FALSE,ierr)<br>
                          >>>><br>
                          >>>><br>
                          >>>> idxn(1)=0<br>
                          >>>> idxm(1)=n-1<br>
                          >>>> if
                          ((idxn(1).ge.nl1).and.(idxn(1)<wbr>.le.nl2-1)) then<br>
                          >>>> v(1)=2d0<br>
                          >>>> call
                          MatSetValues(A,1,idxn,1,idxm,
                          v,INSERT_VALUES,ierr)<br>
                          >>>> end if<br>
                          >>>> call
                          MatAssemblyBegin(A,MAT_FINAL_A<wbr>SSEMBLY,ierr)<br>
                          >>>> call
                          MatAssemblyEnd(A,MAT_FINAL_ASS<wbr>EMBLY,ierr)<br>
                          >>>><br>
                          >>>> if
                          ((idxn(1).ge.nl1).and.(idxn(1)<wbr>.le.nl2-1)) then<br>
                          >>>> v(1)=2d0<br>
                          >>>> call
                          MatGetValues(A,1,idxn,1,idxm, v,ierr)<br>
                          >>>> write(6,*) v<br>
                          >>>> end if<br>
                          >>>><br>
                          >>>> call PetscFinalize(ierr)<br>
                          >>>><br>
                          >>>> end program newnonzero<br>
                          >>>><br>
                          >>>><br>
                          >>>><br>
                          >>>> $ mpiexec.hydra -n 3 ./a.out<br>
                          >>>> [0]PETSC ERROR:
                          --------------------- Error Message
                          ------------------------------<wbr>------------------------------<wbr>--<br>
                          >>>> [0]PETSC ERROR: Argument out
                          of range<br>
                          >>>> [0]PETSC ERROR: Inserting a
                          new nonzero at global row/column (0, 2) into
                          matrix</div>
                      </div>
                      >>>> [0]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html%5Bhttp://www.mcs.anl.gov/petsc/documentation/faq.html%5D" target="_blank">http://www.mcs.anl.gov/petsc/d<wbr>ocumentation/faq.html[http://w<wbr>ww.mcs.anl.gov/petsc/documenta<wbr>tion/faq.html]</a>
                      for trouble shooting.<br>
                      <span class="gmail-m_-3820402716620242094m_-1671753951163694093im gmail-m_-3820402716620242094m_-1671753951163694093HOEnZb">>>>> [0]PETSC
                        ERROR: Petsc Release Version 3.9.2, May, 20,
                        2018<br>
                        >>>> [0]PETSC ERROR: ./a.out on a
                        named tono-hpc1 by marius Wed May 30 09:42:40
                        2018<br>
                        >>>> [0]PETSC ERROR: Configure
                        options --prefix=/home/marius/prog/pet<wbr>sc/3.9.2
                        --download-elemental=yes --download-metis=yes
                        --download-parmetis=yes --download-mumps=yes
--with-scalapack-lib="/home/ma<wbr>rius/intel/compilers_and_libra<wbr>ries_2018.2.199/linux/mkl/lib/<wbr>intel64/libmkl_scalapack_lp64.<wbr>a
                        -Wl,--start-group
/home/marius/intel/compilers_a<wbr>nd_libraries_2018.2.199/linux/<wbr>mkl/lib/intel64/libmkl_intel_<wbr>lp64.a
/home/marius/intel/compilers_a<wbr>nd_libraries_2018.2.199/linux/<wbr>mkl/lib/intel64/libmkl_sequent<wbr>ial.a
/home/marius/intel/compilers_a<wbr>nd_libraries_2018.2.199/linux/<wbr>mkl/lib/intel64/libmkl_core.a
/home/marius/intel/compilers_a<wbr>nd_libraries_2018.2.199/linux/<wbr>mkl/lib/intel64/libmkl_blacs_<wbr>intelmpi_lp64.a
                        -Wl,--end-group -lpthread -lm -ldl"
                        --FC=mpiifort --CC=mpicc --CXX=mpicxx
                        --with-scalar-type=complex --with-mpi-dir=
--with-blaslapack-lib="/home/m<wbr>arius/intel/compilers_and_libr<wbr>aries_2018.2.199/linux/mkl/<wbr>lib/intel64/libmkl_scalapack_<wbr>lp64.a
                        -Wl,--start-group
/home/marius/intel/compilers_a<wbr>nd_libraries_2018.2.199/linux/<wbr>mkl/lib/intel64/libmkl_intel_<wbr>lp64.a
/home/marius/intel/compilers_a<wbr>nd_libraries_2018.2.199/linux/<wbr>mkl/lib/intel64/libmkl_sequent<wbr>ial.a
/home/marius/intel/compilers_a<wbr>nd_libraries_2018.2.199/linux/<wbr>mkl/lib/intel64/libmkl_core.a
/home/marius/intel/compilers_a<wbr>nd_libraries_2018.2.199/linux/<wbr>mkl/lib/intel64/libmkl_blacs_<wbr>intelmpi_lp64.a
                        -Wl,--end-group -lpthread -lm -ldl"
                        --with-cxx-dialect=C++11
                        --download-superlu_dist=yes
                        --download-ptscotch=yes --with-x
                        --with-debugging=1 --download-superlu=yes
                        --with-mkl_cpardiso=1 --with-mkl_pardiso=1
                        --with-scalapack=1<br>
                        >>>> [0]PETSC ERROR: #1
                        MatSetValues_MPIAIJ() line 607 in
                        /home/marius/prog/petsc/petsc-<wbr>3.9.2/src/mat/impls/aij/mpi/mp<wbr>iaij.c<br>
                        >>>> [0]PETSC ERROR: #2
                        MatSetValues() line 1312 in
                        /home/marius/prog/petsc/petsc-<wbr>3.9.2/src/mat/interface/matrix<wbr>.c<br>
                        >>>>
                        (0.000000000000000E+000,0.0000<wbr>00000000000E+000)<br>
                        >>>><br>
                        >>>><br>
                        >>>><br>
                        >>>> Please send complete error
                        message; type of matrix used etc. Ideally code
                        that demonstrates the problem.<br>
                        >>>><br>
                        >>>> Barry<br>
                        >>>><br>
                        >>>></span>
                      <div class="gmail-m_-3820402716620242094m_-1671753951163694093HOEnZb">
                        <div class="gmail-m_-3820402716620242094m_-1671753951163694093h5">>>>>> On May 29,
                          2018, at 3:31 AM, Marius Buerkle <<a href="mailto:mbuerkle@web.de" target="_blank">mbuerkle@web.de</a>[mailto:<a href="mailto:mbuerkle@web.de" target="_blank">mbuerk<wbr>le@web.de</a>]>
                          wrote:<br>
                          >>>>><br>
                          >>>>><br>
                          >>>>> Hi,<br>
                          >>>>><br>
                          >>>>> I tried to set
                          MAT_NEW_NONZERO_LOCATIONS to false, as far as
                          I understood MatSetValues should simply ignore
                          entries which would give rise to new nonzero
                          values not creating a new entry and not cause
                          an error, but I get "[1]PETSC ERROR: Inserting
                          a new nonzero at global row/column". Is this
                          option supposed to work or not?<br>
                          >>>><br>
                          >>><br>
                          >>><br>
                          > <br>
                           </div>
                      </div>
                    </blockquote>
                  </div>
                </div>
              </div>
            </div>
          </div>
        </div></div></div>
      </div>
    </blockquote>
    <p><br>
    </p>
  </div>

</blockquote></div></div></div><br></div></div>
</blockquote></div><br></div></div></div>