[petsc-users] Data types for local ID's and global ID's for large problems

[Jed Brown]

"Weston, Brian Thomas via petsc-users" <petsc-users at mcs.anl.gov> writes:

> Matt,
>
> Thanks for the quick reply. Users of our ALE3D hydro code at LLNL run very large problems , which I believe can pass in large amounts of local integer data to solver packages like HYPRE and PETSc for global solves. All of our local integer data is stored with 32-bit indices and switching to 64-bit indices would increase the memory footprint and data motion. This is a problem when running on our BGQ machines like Sequoia machine and potentially for our GPU machines.
>
> Is this not a performance issue for most PETSc users who are running very large problems on 64-bit indices? And is there any interest in the future for allowing local ID’s to be in 32-bit and global ID’s be in 64-bit?

The reason there is only one PetscInt is to reduce the number of
combinations to be tested (since C has weak typedefs).  The performance
impact observed in practice tends to be quite small.  PETSc might
consider internally changing the storage for common matrix types to use
a separate typedef so that PetscInt would be 64-bit but "PetscMatInt"
could be 32-bit, but I don't think anyone wants that to escape into the
public interface so it wouldn't have any impact on your code, just a
(slight) optimization --with-64-bit-indices.  (Note that some users need
64-bit on a single rank so it needs to be possible to use 64-bit
PetscMatInt.)