<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">Anton:</div><div class="gmail_quote">I am planning to work on this as soon as I get time. I assume that your code is working with the option '<span style="font-size:12.8px">-mat_superlu_dist_fact SamePattern_SameRowPerm'. If not, let me know.</span></div><div class="gmail_quote"><span style="font-size:12.8px"><br></span></div><div class="gmail_quote"><span style="font-size:12.8px">What I'm planing to do is to detect the existence of Pc and Pr in petsc interface, then set reuse option, so users will not be bothered by it.</span></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="#FFFFFF">
    <br>
    Setting Options.Fact = DOFACT for all factorizations is currently
    impossible via PETSc interface. </div></blockquote><div>This might be a bug in our side. I'll check it.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="#FFFFFF">
    The user is expected to choose some kind of reuse model.<br>
    If you could add it, I (and other users probably too) would really
    appreciate that.<br></div></blockquote><div><br></div><div>I'll try to get it done soon, will let you know. Thanks for your patience.</div><div><br></div><div>Hong </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="#FFFFFF"><div><div class="gmail-h5"><br>
    <br>
    <blockquote type="cite">
      <div dir="ltr">
        <div><font face="comic sans ms, sans-serif"><br>
          </font></div>
        <div><font face="comic sans ms, sans-serif">I'll check our
            interface to see if we can add flag-checking for </font><span style="font-family:"comic sans ms",sans-serif">Pr
            and </span><span>Pc, then set default accordingly. </span></div>
        <div><font face="comic sans ms, sans-serif"><br>
          </font></div>
        <div><font face="comic sans ms, sans-serif">Hong<br>
          </font>
          <div class="gmail_extra"><br>
            <div class="gmail_quote">On Wed, Oct 26, 2016 at 3:23 PM,
              Xiaoye S. Li <span dir="ltr"><<a href="mailto:xsli@lbl.gov" target="_blank">xsli@lbl.gov</a>></span> wrote:<br>
              <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                <div dir="ltr">
                  <div>Some graph
                    preprocessing steps can be skipped ONLY IF a
                    previous factorization was done, and the information
                    can be reused (AS INPUT) to the new factorization. <br>
                    <br>
                  </div>
                  <div>In general, the
                    driver routine SRC/pdgssvx.c() performs the LU
                    factorization of the following (preprocessed)
                    matrix:<br>
                     Pc*Pr*diag(R)*A*diag(C)*Pc^T = L*U<br>
                    <br>
                  </div>
                  <div>The default is
                    to do LU from scratch, including all the steps to
                    compute equilibration (R, C), pivot ordering (Pr),
                    and sparsity ordering (Pc). <br>
                    <br>
                  </div>
                  <div>-- The default
                    should be set as options.Fact = DOFACT.<br>
                  </div>
                  <div> <br>
                  </div>
                  <div>-- When you set
                    options.Fact = SamePattern, the sparsity ordering
                    step is skipped, but you need to input Pc which was
                    obtained from a previous factorization.<br>
                    <br>
                  </div>
                  <div>-- When you set
                    options.Fact = SamePattern_SameRowPerm, both
                    sparsity reordering and pivoting ordering steps are
                    skipped, but you need to input both Pr and Pc.<br>
                    <br>
                  </div>
                  <div>Please see
                    Lines 258 - 307 comments in SRC/pdgssvx.c for
                    details, regarding which data structures should be
                    inputs and which are outputs.  The Users Guide also
                    explains this.<br>
                    <br>
                  </div>
                  <div>In EXAMPLE/
                    directory, I have various examples of these usage
                    situations, see EXAMPLE/README.<br>
                    <br>
                  </div>
                  <div>I am a little
                    puzzled why in PETSc, the default is set to
                    SamePattern ??<br>
                    <br>
                  </div>
                  <div>Sherry<br>
                  </div>
                  <div><br>
                  </div>
                </div>
                <div class="gmail-m_8166208793137984007gmail-HOEnZb">
                  <div class="gmail-m_8166208793137984007gmail-h5">
                    <div class="gmail_extra"><br>
                      <div class="gmail_quote">On Tue, Oct 25, 2016 at
                        9:18 AM, Hong <span dir="ltr"><<a href="mailto:hzhang@mcs.anl.gov" target="_blank">hzhang@mcs.anl.gov</a>></span>
                        wrote:<br>
                        <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                          <div dir="ltr">Sherry,
                            <div><br>
                              <div>We set <span style="font-size:12.8px">'-mat_superlu_dist_fact </span><span class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-il" style="background-color:rgb(255,255,255);font-size:12.8px">Sa<wbr>mePattern</span><span style="font-size:12.8px">'  as default
                                  in petsc/superlu_dist on 12/6/15 (see
                                  attached email below). </span></div>
                              <div><span style="font-size:12.8px"><br>
                                </span></div>
                              <div><span style="font-size:12.8px">However,
                                  Anton must set '</span><span style="font-size:12.8px">SamePattern_SameRowPerm'
                                  to avoid crash in his code. Checking </span></div>
                              <div><a href="http://crd-legacy.lbl.gov/%7Exiaoye/SuperLU/superlu_dist_code_html/pzgssvx___a_bglobal_8c.html" target="_blank">http://crd-legacy.lbl.gov/~xia<wbr>oye/SuperLU/superlu_dist_code_<wbr>html/pzgssvx___a_bglobal_8c.ht<wbr>ml</a><br>
                              </div>
                              <div>I see detailed description on using <span style="font-size:12.8px">SamePattern_SameRowPerm,
                                  which requires more from user than </span><span style="font-size:12.8px">SamePattern.
                                  I guess these flags are used for
                                  efficiency. The library sets a
                                  default, then have users to switch for
                                  their own applications. The default
                                  setting should not cause crash. If
                                  crash occurs, give a meaningful error
                                  message would be help.</span></div>
                            </div>
                            <div><span style="font-size:12.8px"><br>
                              </span></div>
                            <div><span style="font-size:12.8px">Do you
                                have suggestion how should we set
                                default in petsc for this flag?</span></div>
                            <div><span style="font-size:12.8px"><br>
                              </span></div>
                            <div><span style="font-size:12.8px">Hong</span></div>
                            <div><span style="font-size:12.8px"><br>
                              </span></div>
                            <div><span style="font-size:12.8px">-------------------</span></div>
                            <div>
                              <div class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-gE gmail-m_-7846754415080799268m_-4562941754056311974gmail-iv gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-gt" style="font-size:12.8px">
                                <table class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-cf gmail-m_-7846754415080799268m_-4562941754056311974gmail-gJ" cellpadding="0">
                                  <tbody>
                                    <tr class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-acZ">
                                      <td class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-gF gmail-m_-7846754415080799268m_-4562941754056311974gmail-gK" style="width:368.8px">
                                        <table class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-cf gmail-m_-7846754415080799268m_-4562941754056311974gmail-ix" style="width:368.8px" cellpadding="0">
                                          <tbody>
                                            <tr>
                                              <td>
                                                <h3 class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-iw"><span name="Hong" class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-gD" style="font-size:12.8px">Hong</span> <span class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-go"><span><</span><a href="mailto:hzhang@mcs.anl.gov" target="_blank">hzhang@mcs.anl.gov</a><span>></span></span></h3>
                                              </td>
                                            </tr>
                                          </tbody>
                                        </table>
                                      </td>
                                      <td class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-gH">
                                        <div class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-gK"><span id="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-:1ue" class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-g3" title="Mon, Dec 7, 2015 at
                                            12:42 PM" alt="Mon, Dec 7,
                                            2015 at 12:42 PM">12/7/15</span>
                                          <div class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-zd" style="outline:0px"><span class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-T-KT"><img class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-f gmail-m_-7846754415080799268m_-4562941754056311974gmail-T-KT-JX" src="https://mail.google.com/mail/u/0/images/cleardot.gif" alt=""></span></div>
                                        </div>
                                      </td>
                                      <td class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-gH"><br>
                                      </td>
                                      <td class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-gH gmail-m_-7846754415080799268m_-4562941754056311974gmail-acX" rowspan="2">
                                        <div class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-T-I gmail-m_-7846754415080799268m_-4562941754056311974gmail-J-J5-Ji gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-T-I-Js-IF gmail-m_-7846754415080799268m_-4562941754056311974gmail-aaq gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-T-I-ax7 gmail-m_-7846754415080799268m_-4562941754056311974gmail-L3"><img class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-hB gmail-m_-7846754415080799268m_-4562941754056311974gmail-T-I-J3" src="https://mail.google.com/mail/u/0/images/cleardot.gif" alt=""></div>
                                        <div id="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-:1r4" class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-T-I gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-J-J5-Ji gmail-m_-7846754415080799268m_-4562941754056311974gmail-T-I-Js-Gs gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-aap gmail-m_-7846754415080799268m_-4562941754056311974gmail-T-I-awG gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-T-I-ax7 gmail-m_-7846754415080799268m_-4562941754056311974gmail-L3"><img class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-hA gmail-m_-7846754415080799268m_-4562941754056311974gmail-T-I-J3" src="https://mail.google.com/mail/u/0/images/cleardot.gif" alt=""></div>
                                      </td>
                                    </tr>
                                    <tr class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-acZ gmail-m_-7846754415080799268m_-4562941754056311974gmail-xD">
                                      <td colspan="3">
                                        <table class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-cf gmail-m_-7846754415080799268m_-4562941754056311974gmail-adz" style="width:448px" cellpadding="0">
                                          <tbody>
                                            <tr>
                                              <td class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-ady">
                                                <div class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-iw gmail-m_-7846754415080799268m_-4562941754056311974gmail-ajw"><span class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-hb">to <span dir="ltr" name="Danyang" class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-g2">Danyang</span>, <span dir="ltr" name="petsc-maint" class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-g2">petsc-maint</span>, <span dir="ltr" name="PETSc" class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-g2">PETSc</span><wbr>, <span dir="ltr" name="Xiaoye" class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-g2">Xiaoye</span></span></div>
                                                <div class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-ajy"><img class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-ajz" id="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-:1r3" src="https://mail.google.com/mail/u/0/images/cleardot.gif" alt=""></div>
                                              </td>
                                            </tr>
                                          </tbody>
                                        </table>
                                      </td>
                                    </tr>
                                  </tbody>
                                </table>
                              </div>
                              <div id="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-:1uc" class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-ii gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-gt gmail-m_-7846754415080799268m_-4562941754056311974gmail-adP gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-adO" style="font-size:12.8px">
                                <div id="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-:1ub" class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-a3s gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-aXjCH gmail-m_-7846754415080799268m_-4562941754056311974gmail-m1517dc034049d992">
                                  <div dir="ltr">
                                    <div class="gmail_extra">
                                      <div class="gmail_quote">Danyang :</div>
                                      <div class="gmail_quote"><br>
                                      </div>
                                      <div class="gmail_quote">Adding
                                        '-mat_superlu_dist_fact <span class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-il">SamePa<wbr>ttern</span>'
                                        fixed the problem. Below is how
                                        I figured it out.</div>
                                      <div class="gmail_quote"><br>
                                      </div>
                                      <div class="gmail_quote">
                                        <div class="gmail_quote" style="font-size:12.8px">1.
                                          Reading ex52f.F, I see
                                          '-superlu_default' =
                                          '-pc_factor_mat_solver_package
                                          superlu_dist', the later
                                          enables runtime options for
                                          other packages. I use
                                          superlu_dist-4.2 and
                                          superlu-4.1 for the tests
                                          below.</div>
                                        <div>...</div>
                                        <div>
                                          <div class="gmail_quote" style="font-size:12.8px">5.</div>
                                          <div class="gmail_quote" style="font-size:12.8px">Using
                                            a_flow_check_1.bin, I am
                                            able to reproduce the error
                                            you reported: all packages
                                            give correct results except
                                            superlu_dist:</div>
                                          <div class="gmail_quote" style="font-size:12.8px">./ex52f
                                            -f0
                                            matrix_and_rhs_bin/a_flow_chec<wbr>k_1.bin
                                            -rhs
                                            matrix_and_rhs_bin/b_flow_chec<wbr>k_168.bin
                                            -loop_matrices flow_check
                                            -loop_folder
                                            matrix_and_rhs_bin -pc_type
                                            lu
                                            -pc_factor_mat_solver_package
                                            superlu_dist</div>
                                          <div class="gmail_quote" style="font-size:12.8px">Norm
                                            of error  2.5970E-12
                                            iterations     1</div>
                                          <span class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-im" style="font-size:12.8px">
                                            <div class="gmail_quote"> -->Test
                                              for matrix          168</div>
                                          </span>
                                          <div class="gmail_quote" style="font-size:12.8px">Norm
                                            of error  1.3936E-01
                                            iterations    34</div>
                                          <span class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-im" style="font-size:12.8px">
                                            <div class="gmail_quote"> -->Test
                                              for matrix          169</div>
                                            <div class="gmail_quote"><br>
                                            </div>
                                          </span>
                                          <div class="gmail_quote" style="font-size:12.8px">I
                                            guess the error might come
                                            from reuse of matrix factor.
                                            Replacing default</div>
                                          <div class="gmail_quote" style="font-size:12.8px">-mat_superlu_dist_fact
                                            <<span class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-il">SamePattern_SameRowPerm</span>>
                                            with</div>
                                          <div class="gmail_quote" style="font-size:12.8px">-mat_superlu_dist_fact <span class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-il">SamePat<wbr>tern</span>,
                                            I get</div>
                                          <div class="gmail_quote" style="font-size:12.8px"><br>
                                          </div>
                                          <div class="gmail_quote" style="font-size:12.8px">./ex52f
                                            -f0
                                            matrix_and_rhs_bin/a_flow_chec<wbr>k_1.bin
                                            -rhs
                                            matrix_and_rhs_bin/b_flow_chec<wbr>k_168.bin
                                            -loop_matrices flow_check
                                            -loop_folder
                                            matrix_and_rhs_bin -pc_type
                                            lu
                                            -pc_factor_mat_solver_package
                                            superlu_dist
                                            -mat_superlu_dist_fact <span class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-il">SamePat<wbr>tern</span></div>
                                          <div class="gmail_quote" style="font-size:12.8px"><br>
                                          </div>
                                          <div class="gmail_quote" style="font-size:12.8px">Norm
                                            of error  2.5970E-12
                                            iterations     1</div>
                                          <span class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-im" style="font-size:12.8px">
                                            <div class="gmail_quote"> -->Test
                                              for matrix          168</div>
                                            <div class="gmail_quote">...</div>
                                            <div class="gmail_quote">
                                              <div class="gmail_quote" style="font-size:12.8px">Sherry
                                                may tell you why <span class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-il">SamePattern_SameRowPerm</span> ca<wbr>use
                                                the difference here.</div>
                                              <div class="gmail_quote" style="font-size:12.8px">Best
                                                on the above
                                                experiments, I would set
                                                following as default</div>
                                              <div class="gmail_quote" style="font-size:12.8px">'-mat_superlu_diagpivotthresh
                                                0.0' in petsc/superlu
                                                interface.</div>
                                              <div class="gmail_quote" style="font-size:12.8px">'-mat_superlu_dist_fact <span class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974gmail-il">SamePa<wbr>ttern</span>'
                                                in petsc/superlu_dist
                                                interface.</div>
                                              <span class="gmail-m_8166208793137984007gmail-m_-7846754415080799268HOEnZb"><font color="#888888">
                                                  <div class="gmail_quote" style="font-size:12.8px"><br>
                                                  </div>
                                                  <div class="gmail_quote" style="font-size:12.8px">Hong</div>
                                                </font></span></div>
                                          </span></div>
                                      </div>
                                    </div>
                                  </div>
                                </div>
                              </div>
                            </div>
                          </div>
                          <div class="gmail-m_8166208793137984007gmail-m_-7846754415080799268HOEnZb">
                            <div class="gmail-m_8166208793137984007gmail-m_-7846754415080799268h5">
                              <div class="gmail_extra"><br>
                                <div class="gmail_quote">On Tue, Oct 25,
                                  2016 at 10:38 AM, Hong <span dir="ltr"><<a href="mailto:hzhang@mcs.anl.gov" target="_blank">hzhang@mcs.anl.gov</a>></span>
                                  wrote:<br>
                                  <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                                    <div dir="ltr">Anton,
                                      <div>I guess, when you reuse
                                        matrix and its symbolic factor
                                        with updated numerical values,
                                        superlu_dist requires this
                                        option. I'm cc'ing Sherry to
                                        confirm it.</div>
                                      <div><br>
                                      </div>
                                      <div>I'll check petsc/superlu-dist
                                        interface to set this flag for
                                        this case.</div>
                                      <span class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974HOEnZb"><font color="#888888">
                                          <div><br>
                                          </div>
                                        </font></span>
                                      <div><span class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974HOEnZb"><font color="#888888">Hong</font></span>
                                        <div>
                                          <div class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974h5"><br>
                                            <div class="gmail_extra"><br>
                                              <div class="gmail_quote">On
                                                Tue, Oct 25, 2016 at
                                                8:20 AM, Anton Popov <span dir="ltr"><<a href="mailto:popov@uni-mainz.de" target="_blank">popov@uni-mainz.de</a>></span>
                                                wrote:<br>
                                                <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                                                  <div bgcolor="#FFFFFF">
                                                    <p>Hong,</p>
                                                    <p>I get all the
                                                      problems gone and
                                                      valgrind-clean
                                                      output if I
                                                      specify this:</p>
                                                    <p>-mat_superlu_dist_fact
SamePattern_SameRowPerm<br>
                                                    </p>
                                                    What does
                                                    SamePattern_SameRowPerm
                                                    actually mean? <br>
                                                    Row permutations are
                                                    for large diagonal,
                                                    column permutations
                                                    are for sparsity,
                                                    right?<br>
                                                    Will it skip
                                                    subsequent matrix
                                                    permutations for
                                                    large diagonal even
                                                    if matrix values
                                                    change
                                                    significantly?<br>
                                                    <br>
                                                    Surprisingly
                                                    everything works
                                                    even with:<br>
                                                    <br>
-mat_superlu_dist_colperm PARMETIS<br>
-mat_superlu_dist_parsymbfact TRUE<br>
                                                    <br>
                                                    Thanks,<br>
                                                    Anton<br>
                                                    <br>
                                                    <div class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974m_-2683123818943114666m_-1905162172435090326moz-cite-prefix">On
                                                      10/24/2016 09:06
                                                      PM, Hong wrote:<br>
                                                    </div>
                                                    <blockquote type="cite">
                                                      <div dir="ltr">
                                                        <div class="gmail_extra">
                                                          <div class="gmail_quote">Anton:
                                                          <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                                                          <div bgcolor="#FFFFFF"><span class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974m_-2683123818943114666m_-1905162172435090326gmail-">
                                                          <blockquote type="cite">
                                                          <div dir="ltr">
                                                          <div class="gmail_extra">
                                                          <div class="gmail_quote">If
                                                          replacing
                                                          superlu_dist
                                                          with mumps,
                                                          does your code
                                                          work?</div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </span> yes</div>
                                                          </blockquote>
                                                          <div> </div>
                                                          <div>You may
                                                          use mumps in
                                                          your code, or
                                                          tests
                                                          different
                                                          options for
                                                          superlu_dist:</div>
                                                          <div><br>
                                                          </div>
                                                          <div>
                                                          <div> 
                                                          -mat_superlu_dist_equil:
                                                          <TRUE>
                                                          Equilibrate
                                                          matrix (None)</div>
                                                          <div> 
                                                          -mat_superlu_dist_rowperm
<LargeDiag> Row permutation (choose one of) LargeDiag NATURAL
                                                          (None)</div>
                                                          <div> 
                                                          -mat_superlu_dist_colperm
<METIS_AT_PLUS_A> Column permutation (choose one of) NATURAL
                                                          MMD_AT_PLUS_A
                                                          MMD_ATA
                                                          METIS_AT_PLUS_A
                                                          PARMETIS
                                                          (None)</div>
                                                          <div> 
                                                          -mat_superlu_dist_replacetinyp<wbr>ivot:
                                                          <FALSE>
                                                          Replace tiny
                                                          pivots (None)</div>
                                                          <div> 
                                                          -mat_superlu_dist_parsymbfact:
                                                          <FALSE>
                                                          Parallel
                                                          symbolic
                                                          factorization
                                                          (None)</div>
                                                          <div> 
                                                          -mat_superlu_dist_fact
<SamePattern> Sparsity pattern for repeated matrix factorization
                                                          (choose one
                                                          of)
                                                          SamePattern
                                                          SamePattern_SameRowPerm
                                                          (None)</div>
                                                          </div>
                                                          <div><br>
                                                          </div>
                                                          <div>The
                                                          options inside
                                                          <> are
                                                          defaults. You
                                                          may try
                                                          others. This
                                                          might help
                                                          narrow down
                                                          the bug.</div>
                                                          <div><br>
                                                          </div>
                                                          <div>Hong</div>
                                                          <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                                                          <div bgcolor="#FFFFFF"><span class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974m_-2683123818943114666m_-1905162172435090326gmail-"><br>
                                                          <blockquote type="cite">
                                                          <div dir="ltr">
                                                          <div class="gmail_extra">
                                                          <div class="gmail_quote">Hong
                                                          <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                                                          <div bgcolor="#FFFFFF"><span class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974m_-2683123818943114666m_-1905162172435090326gmail-m_-8985211249997465386gmail-">
                                                          <div class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974m_-2683123818943114666m_-1905162172435090326gmail-m_-8985211249997465386gmail-m_4450561754203495905moz-cite-prefix">On
                                                          10/24/2016
                                                          05:47 PM, Hong
                                                          wrote:<br>
                                                          </div>
                                                          <blockquote type="cite">
                                                          <div dir="ltr">Barry,
                                                          <div>Your
                                                          change indeed
                                                          fixed the
                                                          error of his
                                                          testing code.</div>
                                                          <div>As Satish
                                                          tested, on
                                                          your branch,
                                                          ex16 runs
                                                          smooth.</div>
                                                          <div><br>
                                                          </div>
                                                          <div>I do not
                                                          understand why
                                                          on maint or
                                                          master branch,
                                                          ex16 creases
                                                          inside
                                                          superlu_dist,
                                                          but not with
                                                          mumps. </div>
                                                          <div><br>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          <br>
                                                          </span> I also
                                                          confirm that
                                                          ex16 runs fine
                                                          with latest
                                                          fix, but
                                                          unfortunately
                                                          not my code.<br>
                                                          <br>
                                                          This is
                                                          something to
                                                          be expected,
                                                          since my code
                                                          preallocates
                                                          once in the
                                                          beginning. So
                                                          there is no
                                                          way it can be
                                                          affected by
                                                          multiple
                                                          preallocations.
                                                          Subsequently I
                                                          only do matrix
                                                          assembly, that
                                                          makes sure
                                                          structure
                                                          doesn't change
                                                          (set to get
                                                          error
                                                          otherwise).<br>
                                                          <br>
                                                          Summary: we
                                                          don't have a
                                                          simple test
                                                          code to debug
                                                          superlu issue
                                                          anymore.<span class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974m_-2683123818943114666m_-1905162172435090326gmail-m_-8985211249997465386gmail-HOEnZb"><font color="#888888"><br>
                                                          <br>
                                                          Anton</font></span><span class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974m_-2683123818943114666m_-1905162172435090326gmail-m_-8985211249997465386gmail-"><br>
                                                          <br>
                                                          <blockquote type="cite">
                                                          <div dir="ltr">
                                                          <div>Hong<br>
                                                          <div class="gmail_extra"><br>
                                                          <div class="gmail_quote">On
                                                          Mon, Oct 24,
                                                          2016 at 9:34
                                                          AM, Satish
                                                          Balay <span dir="ltr"><<a href="mailto:balay@mcs.anl.gov" target="_blank">balay@mcs.anl.gov</a>></span>
                                                          wrote:<br>
                                                          <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span>On
                                                          Mon, 24 Oct
                                                          2016, Barry
                                                          Smith wrote:<br>
                                                          <br>
                                                          ><br>
                                                          </span><span>>
                                                          > [Or
                                                          perhaps Hong
                                                          is using a
                                                          different test
                                                          code and is
                                                          observing bugs<br>
                                                          > > with
                                                          superlu_dist
                                                          interface..]<br>
                                                          ><br>
                                                          >    She
                                                          states that
                                                          her test does
                                                          a NEW
                                                          MatCreate()
                                                          for each
                                                          matrix load (I
                                                          cut and pasted
                                                          it in the
                                                          email I just
                                                          sent). The bug
                                                          I fixed was
                                                          only related
                                                          to using the
                                                          SAME matrix
                                                          from one
                                                          MatLoad() in
                                                          another
                                                          MatLoad().<br>
                                                          <br>
                                                          </span>Ah -
                                                          ok.. Sorry -
                                                          wasn't
                                                          thinking
                                                          clearly :(<br>
                                                          <span class="gmail-m_8166208793137984007gmail-m_-7846754415080799268m_-4562941754056311974m_-2683123818943114666m_-1905162172435090326gmail-m_-8985211249997465386gmail-m_4450561754203495905HOEnZb"><font color="#888888"><br>
                                                          Satish<br>
                                                          </font></span></blockquote>
                                                          </div>
                                                          <br>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          <br>
                                                          </span></div>
                                                          </blockquote>
                                                          </div>
                                                          <br>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          <br>
                                                          </span></div>
                                                          </blockquote>
                                                          </div>
                                                          <br>
                                                        </div>
                                                      </div>
                                                    </blockquote>
                                                    <br>
                                                  </div>
                                                </blockquote>
                                              </div>
                                              <br>
                                            </div>
                                          </div>
                                        </div>
                                      </div>
                                    </div>
                                  </blockquote>
                                </div>
                                <br>
                              </div>
                            </div>
                          </div>
                        </blockquote>
                      </div>
                      <br>
                    </div>
                  </div>
                </div>
              </blockquote>
            </div>
            <br>
          </div>
        </div>
      </div>
    </blockquote>
    <br>
  </div></div></div>

</blockquote></div><br></div></div>