[petsc-dev] PetscUse/TryMethod

Sun Apr 3 12:02:02 CDT 2022

Sundials, for example.

PETSc is still relatively low in the software stack. If everyone is making biannual releases for ECP, then we'd need a topological sort on dependencies and PETSc would need to release (or at least freeze) early, e.g., January or February, so other packages have time to update by their March deadlines.

I understand there may have been an assessment that PetscTryMethod sees vanishing use by dependencies. I think if we're doing that, it should probably be earlier in the release cycle with more effort to assess and notify such dependencies. Maybe we could keep a list of high profile packages, a script to grep them all, and a weekly(?) CI job that builds them.

On Sun, Apr 3, 2022, at 10:45 AM, Barry Smith wrote:
> 
> 
> > On Apr 3, 2022, at 12:24 PM, Satish Balay <balay at mcs.anl.gov> wrote:
> > 
> >> If we had this attitude with the external packages PETSc uses we would have to stop using most of the packages/*.py. 
> > 
> > Sure one can take extreme view on both sides. [no change, vs won't hesitate to change] - having a manageable (minimal) change is harder to do.
> > 
> > I would point out that most externalpackages don't change much and we benefit from it - hence we are able to support so may. Some packages had major changes - and we haven't upgraded to their new versions.
> 
>   What packages are these?  We should have a tool that runs through all the packages/xxx.py and determines the date of release of the version we are using and if there are any newer versions available. We could run this tool automatically a month before each PETSc release sending its output to petsc-dev to see what we should be updating. 
> 
>   Note also that some packages we don't update to, not because of API changes but because the new releases are broken in some way, this is life in the HPC world.
> 
> 
> 
> 
> > 
> > [i.e with the current state one can use them only if they completely buy into petsc ecosystem i.e use old version - but not any larger one - as in use newer features from them]
> > 
> > We did update to newer interfaces in some packages.
> > 
> > But these problems remain - and have to be dealt with - and sometimes the complexity increases based on the dependency tree.
> > 
> > [and also results in folk using and requiring help with older  petsc versions]
> > 
> > Satish
> > 
> > 
> > On Sun, 3 Apr 2022, Barry Smith wrote:
> > 
> >> 
> >>  I would say it is not reasonable for the package developers in the xsdk ecosystem to expect that they can just continue to use another HPC package for multiple years without doing some minimal amount of work to keep up with the other packages' new releases. If we had this attitude with the external packages PETSc uses we would have to stop using most of the packages/*.py. Yes, it is a constant race to keep up the versions in packages/*.py and requires some effort but if you want to play in this game that is a race you have to remain in. And it goes way beyond HPC, to say you do software development but don't need to manage constant change in everything is an oxymoron. There was never a golden age of computing where things didn't change rapidly, pretending there was or can be is not productive. Of course, we want to minimize public change, but having a goal of no public change is not a realistic or even desirable goal.
> >> 
> >>> Just noticed - CHKERRQ() got removed from fortran interface - breaking pflotran
> >> 
> >> This was just a oversight, easily fixed. 
> >> 
> >>> On Apr 3, 2022, at 11:13 AM, Satish Balay <balay at mcs.anl.gov> wrote:
> >>> 
> >>> 
> >>> Note this  is not just 'users should update their code' issue.
> >>> - all packages (that use petsc) would need to do this update
> >>> - and this update doesn't always happen - so pakages will stay at old release - some might not
> >>> - so now we cant build PETSc with both these packages together.
> >>> 
> >>> this type of change causes major issues in xsdk ecosystem (depends on how many direct/indirect dependencies are on the given package)
> >>> 
> >>> Just noticed - CHKERRQ() got removed from fortran interface - breaking pflotran
> >>> 
> >>> https://gitlab.com/xsdk-project/spack-xsdk/-/jobs/2285145624
> >>> 
> >>> [also CHKERRABORT]. Perhaps they can be added back in.
> >>> 
> >>> $ git diff release-3.16..release include/petsc/finclude/petscsys.h
> >>> diff --git a/include/petsc/finclude/petscsys.h b/include/petsc/finclude/petscsys.h
> >>> <snip>
> >>> #define SETERRABORT(c,ierr,s)  call PetscError(c,ierr,0,s); call MPI_Abort(c,ierr)
> >>> -#define CHKERRQ(ierr) if (ierr .ne. 0) then;call PetscErrorF(ierr);return;endif
> >>> +#define PetscCall(ierr) if (ierr .ne. 0) then;call PetscErrorF(ierr);return;endif
> >>> #define CHKERRA(ierr) if (ierr .ne. 0) then;call PetscErrorF(ierr);call MPIU_Abort(PETSC_COMM_SELF,ierr);endif
> >>> -#define CHKERRABORT(c,ierr) if (ierr .ne. 0) then;call PetscErrorF(ierr);call MPI_Abort(c,ierr);endif
> >>> +#define PetscCallAbort(c,ierr) if (ierr .ne. 0) then;call PetscErrorF(ierr);call MPI_Abort(c,ierr);endif
> >>> #define CHKMEMQ call chkmemfortran(__LINE__,__FILE__,ierr)
> >>> 
> >>> Satish
> >>> 
> >>> On Sun, 3 Apr 2022, Barry Smith wrote:
> >>> 
> >>>> 
> >>>>  To use the latest version of PETSc, each user needs to remove the error checks on these calls. The resulting code will work with previous versions of PETSc as well as the current version of PETSc.  PETSc has never promised complete backward compatibility in the sense of promising that one can use new PETSc releases without any changes to their code; the documentation has always stated new releases will contain changes in the API. We began using depreciate a few years ago to limit the number of changes that needed to be made immediately for each release but depreciate is not suitable for all changes and so users do need to make some changes for each new release. 
> >>>> 
> >>>> 
> >>>> 
> >>>> 
> >>>> 
> >>>>> On Apr 3, 2022, at 7:23 AM, Lisandro Dalcin <dalcinl at gmail.com> wrote:
> >>>>> 
> >>>>> The recent PetscUse/TryMethod changes are backward incompatible. Third-party codes cannot compile without modification. Our users deserve better.
> >>>>> 
> >>>>> 
> >>>>> -- 
> >>>>> Lisandro Dalcin
> >>>>> ============
> >>>>> Senior Research Scientist
> >>>>> Extreme Computing Research Center (ECRC)
> >>>>> King Abdullah University of Science and Technology (KAUST)
> >>>>> http://ecrc.kaust.edu.sa/ <http://ecrc.kaust.edu.sa/>
> >>>> 
> >>>> 
> >>> 
> >> 
> > 
> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20220403/d29e5dc8/attachment.html>