<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <br>
    <div class="moz-cite-prefix">On 21/2/2018 9:00 AM, Matthew Knepley
      wrote:<br>
    </div>
    <blockquote type="cite"
cite="mid:CAMYG4G=PV5JFMcVZkWG+SNOwuc+FU_ot4HUksUHPKs+DMmL0og@mail.gmail.com">
      <div dir="ltr">
        <div class="gmail_extra">
          <div class="gmail_quote">On Tue, Feb 20, 2018 at 7:54 PM, TAY
            wee-beng <span dir="ltr"><<a
                href="mailto:zonexo@gmail.com" target="_blank"
                moz-do-not-send="true">zonexo@gmail.com</a>></span>
            wrote:<br>
            <blockquote class="gmail_quote" style="margin:0 0 0
              .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br>
              <br>
              When I run my CFD code with a grid size of 1119x1119x499 (
              total grid size =    624828339 ), I got the error saying I
              need to compile PETSc with 64-bit indices.<br>
              <br>
              So I tried to compile PETSc again and then compile my CFD
              code with the newly compiled PETSc. However, now I got
              segmentation error:<br>
              <br>
              rm: cannot remove `log': No such file or directory<br>
              [409]PETSC ERROR: ------------------------------<wbr>------------------------------<wbr>------------<br>
              [409]PETSC ERROR: [535]PETSC ERROR: [410]PETSC ERROR:
              ------------------------------<wbr>------------------------------<wbr>------------<br>
              [410]PETSC ERROR: Caught signal number 11 SEGV:
              Segmentation Violation, probably memory access out of
              range<br>
              [410]PETSC ERROR: Try option -start_in_debugger or
              -on_error_attach_debugger<br>
              [410]PETSC ERROR: [536]PETSC ERROR:
              ------------------------------<wbr>------------------------------<wbr>------------<br>
              [536]PETSC ERROR: Caught signal number 11 SEGV:
              Segmentation Violation, probably memory access out of
              range<br>
              [536]PETSC ERROR: Try option -start_in_debugger or
              -on_error_attach_debugger<br>
              [536]PETSC ERROR: or see <a
                href="http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind"
                rel="noreferrer" target="_blank" moz-do-not-send="true">http://www.mcs.anl.gov/petsc/d<wbr>ocumentation/faq.html#valgrind</a><br>
              [536]PETSC ERROR: or try <a href="http://valgrind.org"
                rel="noreferrer" target="_blank" moz-do-not-send="true">http://valgrind.org</a>
              on GNU/linux and Apple Mac OS X to find memory corruption
              errors<br>
              [536]PETSC ERROR: likely location of problem given in
              stack below<br>
              [536]PETSC ERROR: ---------------------  Stack Frames
              ------------------------------<wbr>------<br>
              [536]PETSC ERROR: Note: The EXACT line numbers in the
              stack are not available,<br>
              [536]PETSC ERROR:       INSTEAD the line number of the
              start of the function<br>
              [536]PETSC ERROR:       is given.<br>
              [536]PETSC ERROR: [536] DMDACheckOwnershipRanges_Priva<wbr>te
              line 581 /home/users/nus/tsltaywb/sourc<wbr>e/petsc-3.7.6/src/dm/impls/da/<wbr>da.c<br>
              [536]PETSC ERROR: or see <a
                href="http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind"
                rel="noreferrer" target="_blank" moz-do-not-send="true">http://www.mcs.anl.gov/petsc/d<wbr>ocumentation/faq.html#valgrind</a><br>
              [410]PETSC ERROR: or try <a href="http://valgrind.org"
                rel="noreferrer" target="_blank" moz-do-not-send="true">http://valgrind.org</a>
              on GNU/linux and Apple Mac OS X to find memory corruption
              errors<br>
              [410]PETSC ERROR: likely location of problem given in
              stack below<br>
              [410]PETSC ERROR: ---------------------  Stack Frames
              ------------------------------<wbr>------<br>
              [410]PETSC ERROR: Note: The EXACT line numbers in the
              stack are not available,<br>
              [897]PETSC ERROR: [536] DMDASetOwnershipRanges line 613
              /home/users/nus/tsltaywb/sourc<wbr>e/petsc-3.7.6/src/dm/impls/da/<wbr>da.c<br>
              [536]PETSC ERROR: [536] DMDACreate3d line 1434
              /home/users/nus/tsltaywb/sourc<wbr>e/petsc-3.7.6/src/dm/impls/da/<wbr>da3.c<br>
              [536]PETSC ERROR: --------------------- Error Message
              ------------------------------<wbr>------------------------------<wbr>--<br>
              <br>
              The CFD code worked previously but increasing the problem
              size results in segmentation error. It seems to be related
              to DMDACreate3d and DMDASetOwnershipRanges. Any idea where
              the problem lies?<br>
              <br>
              Besides, I want to know when and why do I have to use
              PETSc with 64-bit indices?<br>
            </blockquote>
            <div><br>
            </div>
            <div>1) A 32-bit integer can hold numbers up to 2^32 =
              4.2e9, so if you have a 3D velocity, pressure, and energy,
              you already have 3e9 unknowns,</div>
            <div>    before you even start to count nonzero entries in
              the matrix. 64-bit integers allow you to handle these big
              sizes.</div>
            <div> </div>
            <blockquote class="gmail_quote" style="margin:0 0 0
              .8ex;border-left:1px #ccc solid;padding-left:1ex">
              Also, can I use the 64-bit indices version with smaller
              sized problems?<br>
            </blockquote>
            <div><br>
            </div>
            <div>2) Yes</div>
            <div> </div>
            <blockquote class="gmail_quote" style="margin:0 0 0
              .8ex;border-left:1px #ccc solid;padding-left:1ex">
              And is there a speed difference between using the 32-bit
              and 64-bit indices ver?</blockquote>
            <div><br>
            </div>
            <div>3) I have seen no evidence of this</div>
            <div><br>
            </div>
            <div>4) My guess is that you have defines regular integers
              in your code and passed them to PETSc, rather than using
              PetscInt as the type.</div>
          </div>
        </div>
      </div>
    </blockquote>
    Oh that seems probable. So I am still using integer(4) when it
    should be integer(8) for some values, is that so? If I use PetscInt,
    is it the same as integer(8)? Or does it depend on the actual
    number?<br>
    <br>
    I wonder if I replace all my integer to PetscInt, will there be a
    large increase in memory usage, because all integer(4) now becomes
    integer(8)?<br>
    <br>
    Thanks.<br>
    <blockquote type="cite"
cite="mid:CAMYG4G=PV5JFMcVZkWG+SNOwuc+FU_ot4HUksUHPKs+DMmL0og@mail.gmail.com">
      <div dir="ltr">
        <div class="gmail_extra">
          <div class="gmail_quote">
            <div><br>
            </div>
            <div>  Thanks,</div>
            <div><br>
            </div>
            <div>     Matt</div>
            <div> </div>
            <blockquote class="gmail_quote" style="margin:0 0 0
              .8ex;border-left:1px #ccc solid;padding-left:1ex"><span
                class="HOEnZb"><font color="#888888"><br>
                  -- <br>
                  Thank you very much.<br>
                  <br>
                  Yours sincerely,<br>
                  <br>
                  ==============================<wbr>==================<br>
                  TAY Wee-Beng (Zheng Weiming) 郑伟明<br>
                  Personal research webpage: <a
                    href="http://tayweebeng.wixsite.com/website"
                    rel="noreferrer" target="_blank"
                    moz-do-not-send="true">http://tayweebeng.wixsite.com/<wbr>website</a><br>
                  Youtube research showcase: <a
                    href="https://www.youtube.com/channel/UC72ZHtvQNMpNs2uRTSToiLA"
                    rel="noreferrer" target="_blank"
                    moz-do-not-send="true">https://www.youtube.com/channe<wbr>l/UC72ZHtvQNMpNs2uRTSToiLA</a><br>
                  linkedin: <a
                    href="http://www.linkedin.com/in/tay-weebeng"
                    rel="noreferrer" target="_blank"
                    moz-do-not-send="true">www.linkedin.com/in/tay-weeben<wbr>g</a><br>
                  ==============================<wbr>==================<br>
                  <br>
                </font></span></blockquote>
          </div>
          <br>
          <br clear="all">
          <div><br>
          </div>
          -- <br>
          <div class="gmail_signature" data-smartmail="gmail_signature">
            <div dir="ltr">
              <div>
                <div dir="ltr">
                  <div>What most experimenters take for granted before
                    they begin their experiments is infinitely more
                    interesting than any results to which their
                    experiments lead.<br>
                    -- Norbert Wiener</div>
                  <div><br>
                  </div>
                  <div><a href="http://www.caam.rice.edu/%7Emk51/"
                      target="_blank" moz-do-not-send="true">https://www.cse.buffalo.edu/~knepley/</a><br>
                  </div>
                </div>
              </div>
            </div>
          </div>
        </div>
      </div>
    </blockquote>
    <br>
  </body>
</html>