[petsc-dev] MatSetOption is logically collective even for MAT_ROW_ORIENTED.

[Patrick Lacasse]

Thanks Barry for your answer,

Patrick

2012/9/20 Barry Smith <bsmith at mcs.anl.gov>

>
>    Patrick,
>
>     We may have gone overboard with our assertions in this routine.
>
>     Some MatOptions really should be collective while others like the one
> you are using need not be collective.
>
>     Simple solution just remove those two collective asserts.  Harder
> solution, use the asserts for only the options that should be checked. We
> could for example use less then some number for collective asserts and
> greater than number for for non collective. A little hacky but would work?
>
>    Barry
>
>
>
> On Sep 20, 2012, at 11:42 AM, Patrick Lacasse <patrick.m.lacasse at gmail.com>
> wrote:
>
> > Hi,
> >
> >     I just discovered that the way we assemble our matrix cause a
> deadlock in MatSetOption
> > when petsc is compiled with debugging option enabled.
> >
> > I resume :
> > - We loop over elements.
> > - For each element we create a small elementary matrix.
> > - We assemble the elementary matrix with MatSetValues into the big one.
> >
> > Sometimes, we have some rectangular elementary matrix that needs to be
> added twice,
> > the second time it is transposed. The big matrix is not symmetric for
> many possible reasons.
> > We are essentially doing it for every elementary matrix that needs it :
> >
> > MatSetOption(mat,MAT_ROW_ORIENTED,true)
> > MatSetValues(...)
> > MatSetOption(mat,MAT_ROW_ORIENTED,false)
> > MatSetValues(...)
> >
> > Of course, it fails (deadlock) in MatSetOption at assertion
> > PetscValidLogicalCollectiveEnum(mat,op,2);
> > because different process don't loop over the same (number of) elements.
> > However, it is working fine without the assertion.
> >
> > I have two question :
> > Is there really a risk to call MatSetOption(mat, MAT_ROW_ORIENTED, flg)
> on only one process?
> > What can I do now?
> >
> > Some possible answer are
> > - Close our eyes and stop compiling petsc with debugging option enabled.
> > - We could copy our elementary matrix into a transposed one before to
> MatSetValues. We don't want do this cause we think it would slow down our
> code.
> > - Could MatSetValues has a bonus argument row_oriented?
> > - Could the assertions in MatSetOption be skipped for some options?
> > - Could we add a function MatSetRowOriented which would not be
> collective?
> >
> > thanks,
> >
> > Patrick Lacasse
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20120926/3d1e3710/attachment.html>