[petsc-dev] Adding support memkind allocators in PETSc

[Barry Smith]

> On Jun 3, 2015, at 9:51 PM, Jed Brown <jed at jedbrown.org> wrote:
> 
> Barry Smith <bsmith at mcs.anl.gov> writes:
>>  Perhaps, and this is just nonsense off the top of my head, if you
>>  had some measure of the importance of a vector (or matrix; I would
>>  start with vectors for simplicity and since we have more of them)
>>  based on how often it's values would be "accessed". So a vector that
>>  you know is only used "once in a while" gets a lower "importance"
>>  than one that gets used "very often". Of course determining these
>>  vectors importances may be difficult. You could do it
>>  experimentally, add some code that measures how often each vector
>>  gets its values "accessed (whatever that means)/read write" and see
>>  if there is some distribution (do this for a nontrivial TS example)
>>  where some vectors are accessed often and others rarely. 
> 
> This is what I termed profile-guided and it's very accurate (you have
> global space-time information), but super brittle when
> resource-constrained.

  Sure but the super high-end (DOE LCF centers) focus allows (actually loves)  the need for "super brittle" stuff.  It is not the bread and butter of PETSc but if they (silly ASCR) are willing to foot our bills to do our bread and butter by pandering to the "super brittle" high-end what's the harm in pandering (aside from our souls) since we are not actually doing the work :-)


> 
> Note that in case of Krylov solvers, the first vectors in the Krylov
> space are accessed far more than later vectors (e.g., the 30th vector is
> accessed once per 30 iterations versus the first vector which is
> accessed every iteration).  Simple greedy allocation is great for this
> case.
> 
> It's terrible in other cases, a simple case of which is two solvers
> where the first is cheap (or solved only rarely) and the second is
> solved repeatedly at great expense.  Nested solvers are one such
> example.  But you don't know which one is more expensive except in
> retrospect, and this can even change as nonlinearities evolve.