[petsc-dev] VECCUSP needs to be zeroed?

[Jed Brown]

On Wed, Nov 9, 2011 at 09:57, Lisandro Dalcin <dalcinl at gmail.com> wrote:

> Are you 100% sure about this? What's the point of wasting cycles by
> explicitly calling Memzero in VecCreate_Seq? I bet may codes out there
> (and even PETSc's testsuite) will break if you do not initialize all
> entries to zero! For example, if you create a Vec or Mat and
> immediately enter a loop using {Vec|Mat}SetValues(..., ADD_VALUES), I
> think most people would expect this to work.
>

In practice, they are initialized to prevent constant support requests. If
we intend for it to "mean" uninitialized, then Memzero in VecCreate_Seq()
should be turned off when run under Valgrind, so that we can find places
where the assumption is made.

This is not nearly as simple as it may seem on NUMA. You want to fault the
memory in the same pattern as it will be used later. It would be hard for
Memzero to know how you wanted the memory faulted, so it would be expected
to fault the memory incorrectly. Either we need a way to specify a locality
(maybe as a coloring) so that Memzero can be correctly threaded, or we
should NOT initialize the memory and rely on the user to do that. As much
of a hassle as the latter is, it's still not a complete solution because
all the Vec operations should use threads with the same memory distribution
as mesh traversal.

For the hardware, it is probably sufficient to make the NUMA partition
either full blocks or interleaved at page-size granularity.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20111109/7c114d59/attachment.html>