[petsc-users] Compiling with PETSc 64-bit indices

[Matthew Knepley]

On Tue, Feb 20, 2018 at 7:54 PM, TAY wee-beng <zonexo at gmail.com> wrote:

> Hi,
>
> When I run my CFD code with a grid size of 1119x1119x499 ( total grid size
> =    624828339 ), I got the error saying I need to compile PETSc with
> 64-bit indices.
>
> So I tried to compile PETSc again and then compile my CFD code with the
> newly compiled PETSc. However, now I got segmentation error:
>
> rm: cannot remove `log': No such file or directory
> [409]PETSC ERROR: ------------------------------
> ------------------------------------------
> [409]PETSC ERROR: [535]PETSC ERROR: [410]PETSC ERROR:
> ------------------------------------------------------------------------
> [410]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation,
> probably memory access out of range
> [410]PETSC ERROR: Try option -start_in_debugger or
> -on_error_attach_debugger
> [410]PETSC ERROR: [536]PETSC ERROR: ------------------------------
> ------------------------------------------
> [536]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation,
> probably memory access out of range
> [536]PETSC ERROR: Try option -start_in_debugger or
> -on_error_attach_debugger
> [536]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/d
> ocumentation/faq.html#valgrind
> [536]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac
> OS X to find memory corruption errors
> [536]PETSC ERROR: likely location of problem given in stack below
> [536]PETSC ERROR: ---------------------  Stack Frames
> ------------------------------------
> [536]PETSC ERROR: Note: The EXACT line numbers in the stack are not
> available,
> [536]PETSC ERROR:       INSTEAD the line number of the start of the
> function
> [536]PETSC ERROR:       is given.
> [536]PETSC ERROR: [536] DMDACheckOwnershipRanges_Private line 581
> /home/users/nus/tsltaywb/source/petsc-3.7.6/src/dm/impls/da/da.c
> [536]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/d
> ocumentation/faq.html#valgrind
> [410]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac
> OS X to find memory corruption errors
> [410]PETSC ERROR: likely location of problem given in stack below
> [410]PETSC ERROR: ---------------------  Stack Frames
> ------------------------------------
> [410]PETSC ERROR: Note: The EXACT line numbers in the stack are not
> available,
> [897]PETSC ERROR: [536] DMDASetOwnershipRanges line 613
> /home/users/nus/tsltaywb/source/petsc-3.7.6/src/dm/impls/da/da.c
> [536]PETSC ERROR: [536] DMDACreate3d line 1434
> /home/users/nus/tsltaywb/source/petsc-3.7.6/src/dm/impls/da/da3.c
> [536]PETSC ERROR: --------------------- Error Message
> --------------------------------------------------------------
>
> The CFD code worked previously but increasing the problem size results in
> segmentation error. It seems to be related to DMDACreate3d and
> DMDASetOwnershipRanges. Any idea where the problem lies?
>
> Besides, I want to know when and why do I have to use PETSc with 64-bit
> indices?
>

1) A 32-bit integer can hold numbers up to 2^32 = 4.2e9, so if you have a
3D velocity, pressure, and energy, you already have 3e9 unknowns,
    before you even start to count nonzero entries in the matrix. 64-bit
integers allow you to handle these big sizes.


> Also, can I use the 64-bit indices version with smaller sized problems?
>

2) Yes


> And is there a speed difference between using the 32-bit and 64-bit
> indices ver?


3) I have seen no evidence of this

4) My guess is that you have defines regular integers in your code and
passed them to PETSc, rather than using PetscInt as the type.

  Thanks,

     Matt


>
> --
> Thank you very much.
>
> Yours sincerely,
>
> ================================================
> TAY Wee-Beng (Zheng Weiming) 郑伟明
> Personal research webpage: http://tayweebeng.wixsite.com/website
> Youtube research showcase: https://www.youtube.com/channe
> l/UC72ZHtvQNMpNs2uRTSToiLA
> linkedin: www.linkedin.com/in/tay-weebeng
> ================================================
>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.caam.rice.edu/~mk51/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20180220/1c9c6901/attachment-0001.html>