[petsc-users] Data types for local ID's and global ID's for large problems

[Weston, Brian Thomas]

Matt,

Thanks for the quick reply. Users of our ALE3D hydro code at LLNL run very large problems , which I believe can pass in large amounts of local integer data to solver packages like HYPRE and PETSc for global solves. All of our local integer data is stored with 32-bit indices and switching to 64-bit indices would increase the memory footprint and data motion. This is a problem when running on our BGQ machines like Sequoia machine and potentially for our GPU machines.

Is this not a performance issue for most PETSc users who are running very large problems on 64-bit indices? And is there any interest in the future for allowing local ID’s to be in 32-bit and global ID’s be in 64-bit?

Best,
Brian

From: Matthew Knepley <knepley at gmail.com>
Date: Wednesday, January 9, 2019 at 2:56 PM
To: "Weston, Brian Thomas" <weston8 at llnl.gov>
Cc: PETSc <petsc-users at mcs.anl.gov>
Subject: Re: [petsc-users] Data types for local ID's and global ID's for large problems

On Wed, Jan 9, 2019 at 5:53 PM Weston, Brian Thomas via petsc-users <petsc-users at mcs.anl.gov<mailto:petsc-users at mcs.anl.gov>> wrote:
Does PETSc have different data types for local ID’s and global ID’s? For example, if PETSc is configured for 32-bit indices on very large problems with say 10 billion degrees of freedoms (exceeding the limit of 32-bit indices for a global ID) would it crash and require us to re-compile with 64-bit indices or can it gracefully handle this?

You would need to reconfigure for 64-bit indices. Keeping local indices 32-bit does not do much. I experimented with a local-index-only version several years ago, but it breaks a lot of things, and does not really deliver much benefit.

  Thanks,

    Matt

Sincerely,
Brian


--
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/<http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190110/b4eb6518/attachment.html>