[Petsc-trilinos-discussion] Scope and requirements

[Barry Smith]

  We are not here to debate the merits of any of our choices. Rather we are here to understand the commonalities and differences and see what we can do to address the concerns of the DOE program managers and DOE simulation community. If we argue over who is right and who is wrong and try to convince each other to switch to the “right” way we’ll just go down an unproductive rat hole. 

   PETSc has two mailing lists petsc-maint at mcs.anl.gov (private) and petsc-users at mcs.anl.gov (public) were we can help people with the issues of 1) using PETSc with a separate Trilinos build (in theory this is trivial) 2) initializing PETSc properly for runs on subset of processes (again in theory this is trivial) and 3) how to manage “viewing” information about various sub solvers so that they are not “contaminated” by output from other parts of the code. I hope you can send questions/bug reports to these lists so we can help you sort out these issues. Frankly I am a bit concerned that since they are issues that concern PETSc users we haven’t been contacted to determine how to deal with them, we answers questions from users all over the place who have no relationship to DOE and certainly want to provide the same level of help for DOE users. So please if you have these difficulties with PETSc let us know on either the private or public mailing list and we’ll help you.

   That said, the “interoperability” issues that Roscoe brings up are important in the abstract and we should discuss them (just not as a “you guys are doing it wrong” argument.)


   Barry

On Nov 21, 2013, at 2:50 PM, Jed Brown <jedbrown at mcs.anl.gov> wrote:

> Ross, your email quoting style is nearly unreadable.  Can you please use
> normal nested quoting?
> 
> "Bartlett, Roscoe A." <bartlettra at ornl.gov> writes:
> 
>> Matt, But someone would have to physically modify one or perhaps two
>> of the three APPs in the three-APP scenario mentioned in the below
>> slides and paper right?  If the breaks in backward compatibility are
>> significant, who is going to upgrade APP1 and APP2 to the current
>> version of Library A (or at least the one currently used in APP3)?  If
>> the core developers for APP1 and APP2 are not part of the project to
>> couple these with APP3, then who is going to make those mortifications
>> to APP1 and APP2?  
> 
> The changes should go upstream as a normal part of maintaining APP1 and
> APP2.  The changes are almost always very easy to apply.
> 
>> How can they safely make those changes if they are not the core
>> developers?  Will those mortifications make it back into the master
>> sources for APP1 and APP2?  
> 
> I think you're making this out to be more onerous than it really is.  Do
> the packages you speak of have much more advanced usage than packages
> like libMesh, MOOSE, and PFLOTRAN?
> 
>> If the breaks in backward compatibility are minor, then what is the
>> big deal in maintain backward compatibility over longer periods of
>> time?  
> 
> It is confusing for new users, is something else to test, and clutters
> the implementation.  Strict backward compatibility is expensive and that
> cost has to be balanced relative to maintenance of the packages that use
> PETSc.
> 
>> [Bartlett, Roscoe A.] So PETSc does maintain backward compatibility of
>> the interfaces and behavior?  I think that constitutes the definition
>> of "backward compatibility".  
> 
> PETSc uses encapsulation so that changes to external packages are hidden
> From users.  We maintain source and binary compatibility in subminor
> versions, so 3.4.x is binary compatible.  We promise neither source or
> binary compatibility between feature releases (3.5 has some changes from
> 3.4).  The changes are usually in more advanced interfaces, we document
> them on the website, and the upgrade path is usually very simple.  We
> occasionally offer __attribute((deprecated)) to bridge between feature
> releases, especially among less advanced interfaces, but it is not
> universal.
> 
> Our experience has been that clarity and simplicity for beginners, and
> cleanliness for developers outweigh the minor effort of updating
> applications.
> 
>> As for the code GAMG to replace ML, should we tell the LANL Hydra-TH
>> developers to stop using ML under PETSc and use GAMG instead?  Will
>> they get the same or better performance in all cases?  
> 
> Both are useful.  GAMG can be accessed with -pc_type gamg.
> 
>> [Bartlett, Roscoe A.] Have you seen exodiff-based testing?  Exodiff
>> uses an unscaled L-infinity norm that makes it very hard to set good
>> tolerances.  If applications defined a application-specific inner
>> product with physics meaning, setting a good engineer tolerance would
>> be easy and robust.  Yes, if application codes defined good robust
>> tests this would be a piece of cake but they don't in many cases.
>> That is the reality with these codes.  
> 
> I'm familiar with exodiff tests.  If the tests have quantified
> discretization error, then any solver with an algebraic tolerance that
> is more accurate than discretization error will compare properly.
> 
> The definition of a bad test is one that needs to be updated due to an
> indirect implementation change that still fulfills the interface.  We
> cannot be responsible for accommodating every app's testing mistakes.
> 
>> However, that does not answer the issue of performance of ML vs. GAMG
>> which the Hydra-TH developers and users will care a lot about also
>> obviously.  Googling for "GAMG preconditioner" does not find
>> anything. Should I be searching for something else?  
> 
> It uses essentially the same algorithms as Prometheus.  Access with
> -pc_type gamg, more info in -help, the man pages, and examples.  Maybe
> we will write a technical report later.
> 
>> BTW: The guidelines you list below for smoothing the breaks in
>> backward compatibility are important, but that does *not* negate the
>> advantages in maintaining contiguous sets of releases that maintain
>> backward compatibility when considering larger software ecosystems.
>> Those measures just make it less painful when you do finally break
>> backward compatibility but that change still requires careful manual
>> changes to the code, usually by the core developers of the user code.
> 
> PETSc interface changes are rarely difficult to update across.  We also
> tend to see that if the old interface continues to be available, people
> procrastinate and never update, until the package is ultimately
> discontinued or you remove the backward-compatibility.  It is more
> expensive to defer, so we encourage people to do it right away.  Most
> users have been okay with this.
> 
>> Are you going to the SWP2XS workshop in January?
> 
> I will be there.