[petsc-users] DMLocalToGlobal PETScSF is BROKEN

[Barry Smith]

     Jed and Matt,

      This SF stuff is broken! Please fix.

PetscErrorCode  DMLocalToGlobalBegin(DM dm,Vec l,InsertMode mode,Vec g)
{
  PetscSF        sf;
  PetscErrorCode ierr;

  PetscFunctionBegin;
  PetscValidHeaderSpecific(dm,DM_CLASSID,1);
  ierr = DMGetDefaultSF(dm, &sf);CHKERRQ(ierr);
  if (sf) {
    MPI_Op      op;
    PetscScalar *lArray, *gArray;

    switch (mode) {
    case INSERT_VALUES:
    case INSERT_ALL_VALUES:
      op = MPIU_REPLACE; break;
    case ADD_VALUES:
    case ADD_ALL_VALUES:
      op = MPI_SUM; break;
    default:
      SETERRQ1(PetscObjectComm((PetscObject)dm), PETSC_ERR_ARG_OUTOFRANGE, "Invalid insertion mode %D", mode);
    }
    ierr = VecGetArray(l, &lArray);CHKERRQ(ierr);
    ierr = VecGetArray(g, &gArray);CHKERRQ(ierr);
    ierr = PetscSFReduceBegin(sf, MPIU_SCALAR, lArray, gArray, op);CHKERRQ(ierr);
    ierr = VecRestoreArray(l, &lArray);CHKERRQ(ierr);
    ierr = VecRestoreArray(g, &gArray);CHKERRQ(ierr);

   But the PetscSF stuff has no way of managing multiple begin/begin in flight and no error checking to warn people against it. 

   Likely the fix belongs in maint

   Barry



On Jan 21, 2014, at 10:21 PM, Dharmendar Reddy <dharmareddy84 at gmail.com> wrote:

> Thanks.  I assumed it to work. Can you add a note to man pages ? It
> took me a lot of iterations to identify the incorrect usage as the
> cause.
> 
> I will use CHKERRQ.
> 
> 
> Thanks
> Reddy
> 
> On Tue, Jan 21, 2014 at 10:04 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
>> 
>> On Jan 21, 2014, at 9:52 PM, Dharmendar Reddy <dharmareddy84 at gmail.com> wrote:
>> 
>>> Hello,
>>>        I am trying to understand the usage rule for DMLocalToGlobalBegin/End.
>>> 
>>> I have a sequence of calls like this:
>>> 
>>> call DMLocalToGlobalBegin(dm, localLB, INSERT_VALUES, LB, ierr)    ----- (1)
>>> 
>>> call DMLocalToGlobalBegin(dm, localUB, INSERT_VALUES, UB, ierr)   ------ (2)
>>> 
>>> call DMLocalToGlobalEnd(dm, localLB, INSERT_VALUES, LB, ierr)    ------- (3)
>>> 
>>> call DMLocalToGlobalEnd(dm, localUB, INSERT_VALUES, UB, ierr)    --------(4)
>>> 
>>> 
>>> call SNESVISetVariableBounds(snes,LB,UB, ierr)
>>> 
>>> 
>>> SNES fails to run. I get function norm zero right in the first call
>>> and simulation ends.
>>> 
>>> 
>>> If i change the order of DM calls above to
>>> 
>>> 1,3,2,4 then code works as expected
>>> 
>>> 
>>> Does that mean the End call should occur immediately after Begin call ?
>> 
>>  Currently they do need to be called in that order. Note that if you checked the ierr after (2) it would have indicated an error condition. I am surprised that PETSc didn’t print an error message “Scatter ctx already in use”. Note in FORTRAN you can do a lazy check of ierr with
>> 
>>  call DMLocalToGlobalBegin(dm, localUB, INSERT_VALUES, UB, ierr) ; CHKERRQ(ierr)
>> 
>>  so it does not make the Fortran code particularly ugly just put the CHKERRQ(ierr) on the same line as the code
>> 
>> 
>>  Barry
>> 
>> 
>>> 
>>> 
>>> Thanks
>>> Reddy
>> 
> 
> 
> 
> -- 
> -----------------------------------------------------
> Dharmendar Reddy Palle
> Graduate Student
> Microelectronics Research center,
> University of Texas at Austin,
> 10100 Burnet Road, Bldg. 160
> MER 2.608F, TX 78758-4445
> e-mail: dharmareddy84 at gmail.com
> Phone: +1-512-350-9082
> United States of America.
> Homepage: https://webspace.utexas.edu/~dpr342