[petsc-users] Data types for local ID's and global ID's for large problems

[Matthew Knepley]

On Wed, Jan 9, 2019 at 7:15 PM Jed Brown <jed at jedbrown.org> wrote:

> "Weston, Brian Thomas via petsc-users" <petsc-users at mcs.anl.gov> writes:
>
> > Matt,
> >
> > Thanks for the quick reply. Users of our ALE3D hydro code at LLNL run
> very large problems , which I believe can pass in large amounts of local
> integer data to solver packages like HYPRE and PETSc for global solves. All
> of our local integer data is stored with 32-bit indices and switching to
> 64-bit indices would increase the memory footprint and data motion. This is
> a problem when running on our BGQ machines like Sequoia machine and
> potentially for our GPU machines.
> >
> > Is this not a performance issue for most PETSc users who are running
> very large problems on 64-bit indices? And is there any interest in the
> future for allowing local ID’s to be in 32-bit and global ID’s be in 64-bit?
>
> The reason there is only one PetscInt is to reduce the number of
> combinations to be tested (since C has weak typedefs).  The performance
> impact observed in practice tends to be quite small.  PETSc might
> consider internally changing the storage for common matrix types to use
> a separate typedef so that PetscInt would be 64-bit but "PetscMatInt"
> could be 32-bit, but I don't think anyone wants that to escape into the
> public interface so it wouldn't have any impact on your code, just a
> (slight) optimization --with-64-bit-indices.  (Note that some users need
> 64-bit on a single rank so it needs to be possible to use 64-bit
> PetscMatInt.)
>

As Jed says, all our tests show a negligible difference. However, if you
had tests on BGQ that showed a big impact, we would love to see the
numbers. That could justify work on this.

  Thanks,

     Matt
-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190110/8d8186cd/attachment.html>