[petsc-users] VECMPICUSP with ghosted vector

[Fredrik Heffer Valdmanis]

2012/2/6 Barry Smith <bsmith at mcs.anl.gov>

>
>
>   Fredrik,
>
>    This question belongs on petsc-dev at mcs.anl.gov since it involves
> additions/extensions to PETSc so I am moving the discussion over to there.
>
>    We have not done the required work to have ghosted vectors work with
> CUSP yet, so this will require some additions to PETSc. We can help you
> with that process but since the PETSc team does not have a CUSP person
> developing PETSc full time you will need to actual contribute some code but
> I'll try to guide you in the right direction.
>
>     The first observation is that ghosted vectors in PETSc are actually
> handled with largely the same code as VECMPI vectors (with just no ghost
> points by default) so in theory little work needs to be done to get the
> functionality you need. What makes the needed changes non-trivial is the
> current interface where one calls VecCreateGhost() to create the vectors.
> This is one of our "easy" interfaces and it is somewhat legacy in that
> there is no way to control the types of the vectors since it creates
> everything about the vector in one step.   Note that we have the same
> issues with regard to the pthread versions of the PETSc vectors and
> ghosting.
>
>    So before we even talk about what code to change/add we need to decide
> on the interface.  Presumably you want to be able to decide at runtime
> whether to use regular VECMPI, VECMPICUSP or VECMPIPTHREAD  in your ghosted
> vectors. How do we get that information in there? An additional argument to
> VecCreateGhost() (ugly?)? Options database (by calling VecSetFromOptions()
> ?), other ways?  So for example one could have:
>
> VecCreateGhost(......)
> VecSetFromOptions(......)
>
> to set the specific type cusp or pthread? What about
>
> VecCreateGhost(......)
> VecSetType(......,VECMPICUSP);
>
> which as you note doesn't currently work. Note that the PTHREAD version
> needs to do its own memory allocation so essentially has to undo much of
> what VecCreateGhost() already did, is that a bad thing?
>
> Or do we get rid of VecCreateGhost() completely and change the model to
> something like
>
> VecCreate()
> VecSetType()
> VecSetGhosted()
>
> or
>
> VecCreate()
> VecSetTypeFromOptions()
> VecSetGhosted()
>
> or even
>
> VecCreate()
> VecSetGhosted()   which will just default to regular MPI ghosted.
>
> this model allows a clean implementation that doesn't require undoing
> previously built internals.
>
>
> Everyone chime in with observations so we can figure out any
> refactorizations needed.
>
> Hi Barry,

Thanks for your thorough answer. I am very much interested in contributing.
I am however a little pressed on time for this project, as I am doing it as
part of my master thesis which is due May 1st. I am afraid this might be
out of scope for me, but I am willing to give it a try.

Regarding the user interface, I prefer the suggestions that lie as close to
the original vector interface as possible, ie.

VecCreate()
VecSetType()
VecSetGhosted()

And that VecSetGhosted sets the type to mpi unless the type is already
explicitly set by the user.

I suppose others should give their opinion as well before we decide.

-- 
Fredrik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20120210/e42be4d1/attachment-0001.htm>