[Petsc-trilinos-discussion] Scope and requirements

Thu Nov 21 13:32:53 CST 2013

Matt,

But someone would have to physically modify one or perhaps two of the three APPs in the three-APP scenario mentioned in the below slides and paper right?  If the breaks in backward compatibility are significant, who is going to upgrade APP1 and APP2 to the current version of Library A (or at least the one currently used in APP3)?  If the core developers for APP1 and APP2 are not part of the project to couple these with APP3, then who is going to make those mortifications to APP1 and APP2?  How can  they safely make those changes if they are not the core developers?  Will those mortifications make it back into the master sources for APP1 and APP2?  If the breaks in backward compatibility are minor, then what is the big deal in maintain backward compatibility over longer periods of time?   However, if the breaks in backward compatibility are significant and the behavior of the software changes a lot (even for the better), then that can result in a lot of work to upgrade the (likely poorly designed) test suites of these APP codes.  How can this work in practice?

As for the code GAMG to replace ML, should we tell the LANL Hydra-TH developers to stop using ML under PETSc and use GAMG instead?  Will they get the same or better performance in all cases?  Will this change their solutions requiring them to re-verify their entire test suite?  If the answer is yes to any of these, this will be a harder sell.

BTW: The guidelines you list below for smoothing the breaks in backward compatibility are important, but that does *not* negate the advantages in maintaining contiguous sets of releases that maintain backward compatibility when considering larger software ecosystems.  Those measures just make it less painful when you do finally break backward compatibility but that change still requires careful manual changes to the code, usually by the core developers of the user code.

Cheers,

-Ross

From: Matthew Knepley [mailto:knepley at gmail.com]
Sent: Thursday, November 21, 2013 2:07 PM
To: Bartlett, Roscoe A.
Cc: Jed Brown; Barry Smith; petsc-trilinos-discussion at lists.mcs.anl.gov
Subject: Re: [Petsc-trilinos-discussion] Scope and requirements

On Thu, Nov 21, 2013 at 12:55 PM, Bartlett, Roscoe A. <bartlettra at ornl.gov<mailto:bartlettra at ornl.gov>> wrote:
Matt,

I have read your slides. I think what we propose, using only the latest version, does indeed do what you want:

- Prepare users for the break in backward compatibility
  - We tell the users when we are releasing and post the Changes list months prior. They can also look at the master branch.
    I consider this far better than "deprecated" warnings.

- Fail big and hard when backward compatibility is dropped
  - We do. Configure fails if you do not have what we are looking for.

- Provide for safe straightforward upgrades
  - Since everything is already behind our interface, there are no user code changes.

I would add that in the case of ML its particularly easy, since there is a drop-in replacement (GAMG).

  Thanks,

      Matt

From: Matthew Knepley [mailto:knepley at gmail.com<mailto:knepley at gmail.com>]
Sent: Thursday, November 21, 2013 1:46 PM
To: Bartlett, Roscoe A.
Cc: Jed Brown; Barry Smith; petsc-trilinos-discussion at lists.mcs.anl.gov<mailto:petsc-trilinos-discussion at lists.mcs.anl.gov>

Subject: Re: [Petsc-trilinos-discussion] Scope and requirements

On Thu, Nov 21, 2013 at 12:39 PM, Bartlett, Roscoe A. <bartlettra at ornl.gov<mailto:bartlettra at ornl.gov>> wrote:
> > Therefore, I would say that before there is any talk of more detailed
> > interfaces or interoperability between Trilinos and PETSc that we
> > first solve the basic problems of version compatibility,
>
> PETSc and Trilinos have a different approach to interface evolution.  We
> tend to make minor modifications each year while you tend to create
> entirely new interfaces and coax applications over to the new ones.  The
> need to upgrade inflicts some pain on the users, but our users have
> always ultimately buckled down and done it.
[Bartlett, Roscoe A.]

But the compatibility between Trilinos and PETSc, say in the PETSc/ML case has nothing to do with the "user" (i.e. downstream project using Trilinos and PETSc).  Can they just grab arbitrary versions of PETSC and Trilinos and configure and build them and have the ML support under PETSc work?  If not arbitrary versions (which is unreasonable) what about ranges of versions?  Think about this same issue in a large ecosystem of libraries and applications in a large multi-physics coupled code (CASL VERA is a small example of this we have already had to face).  For example, if every major release of Library A breaks backward compatibility with prior versions and you have to couple together three APPs that all use three different versions of Library A in a statically linked executable, how can that work?  Do you just tell all the APP development teams to all upgrade (or downgrade) to the use same version of Library A that has different interfaces and behavior between the different v
 ersions?  Is that practical?  What if the downstream project development team does even have access to the developers of say APP1 that currently only works against an older version of Library A?  Does the downstream project development team take it upon themselves to upgrade the version of Library A (with new interfaces and behavior) in APP1 that they did not even develop and don't really understand all the test cases for

We support the latest version of a package. Nothing else. Any other strategy is just not feasible.

[Bartlett, Roscoe A.] Supporting the latest version is *not* the issue.  That does not answer the basic question.  How can we expect projects to couple two, three, or more APPs that all use different incompatible versions of the same library? What is the answer?  If a library maintains a longer window of backward compatibility for more mature features, you only have to support the latest version and yet still allow all different kinds of coupling and integration models.  Please keep an open mind and read the balanced proposal for "Regulated Backward Compatibility" in the below reference.  I don't see any other way to achieve broader usage and acceptance of our software in a larger ecosystem of software and projects.  If you see another way, I would like to know what it is.

However, if Library A maintains sufficient backward compatibility between the ranges of versions of Library A currently in use by all three APP codes, then you just need to use the latest version of Library A and that is it.  Easy as pie.  This example, by the way, is shown in slides 36 and 37 in http://web.ornl.gov/~8vt/ORNLCSMDTribitsLifecycleModelPresentation.pdf.  For those interested in the deeper arguments, I would recommend the section on "Regulated Backward Compatibility" in:

    http://web.ornl.gov/~8vt/TribitsLifecycleModel_v1.0.pdf

which also describes the general three APP scenario mentioned above.

Cheers,

-Ross

--
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-trilinos-discussion/attachments/20131121/bdfef3aa/attachment-0001.html>