[petsc-dev] Adding support memkind allocators in PETSc

[Richard Mills]

On Mon, Apr 27, 2015 at 1:56 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:

>
> > On Apr 27, 2015, at 2:46 PM, Jed Brown <jed at jedbrown.org> wrote:
> >
> > Barry Smith <bsmith at mcs.anl.gov> writes:
> >>    MPI_Comm argument?  PETSc users rarely need to call PetscMalloc()
> >>    themselves and if they do call it then they should know the
> >>    properties of the memory they are allocating. Most users won't
> >>    even notice the change.
> >
> > I think that's an exaggeration, but what are you going to use for the
> > "kind" parameter?  The "correct" value depends on a ton of non-local
> > information.
> >
> >>   Note that I'd like to add this argument independent of memkind.
>
>   For things like we talked about the other day. Malloc zones, .... maybe
> some indication that it is ok that the runtime take back this memory if
> available memory is running low, the ability to turn off read or all access
> to a malloc zone so that another library cannot corrupt the data, etc. When
> I said independent of memkind I meant having nothing to do with memkind.
>
>   You are correct that involves lots of nonlocal information or
> information that is not yet known, so the argument cannot be simple flags
> but must be a context that can be modified at later times.  Crude example
>
>   malloc(zone1, n,&x);
>    ....
>   ZoneSetReadOnly(zone1);
>   ..
>   ZoneSetAvailableIfNeeded(zone1);
>
>
I really like Barry's proposal to add this context.  I can think of other
things that could go into that context, too, like hints about how the
memory will be used (passed to the OS via madvise(2), for instance).  Sure,
most of the time people won't want to pass such hints and users could
ignore this, but this is consistent with the PETSc philosophy of exposing
various details to the user if he/she cares, but making a reasonable choice
if the user doesn't.


>
>
>   Barry
>
>
> >
> > What are you going to use it for?  If the allocation is small enough,
> > it'll probably be resident in cache and if it falls out, the lower
> > latency to DRAM will be better than HBM.  As it gets bigger, provided it
> > gets enough use, then HBM becomes the right place, but later it's too
> > big and you have to go back to DRAM.
> >
> > What happens if memory of the kind requested is unavailable?  Error or
> > the implementations tries to find a different kind?  If there are
> > several memory kinds, what order is used when checking?
>
>
The questions in the second paragraph may be something worth enabling the
user to set, either through some global preference or particular entries in
the context.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20150427/9dfd9d5b/attachment.html>