[petsc-dev] PETSc future starting as a new design layer that runs on top of PETSc 3?

[Matthew Knepley]

On Wed, Jul 27, 2022 at 7:25 PM Ham, David A <david.ham at imperial.ac.uk>
wrote:

> This seems like an opportune moment to flag that we’re proposing a sprint
> at some point in 2023 to document petsc4py.  Right now there are no
> docstrings in petsc4py, which means users have to infer what the
> corresponding C function name is and read the docstring for that, then
> understand enough about how the idiom changes between the C and Python
> interfaces to re-interpret the C documentation to Python. This is a pretty
> awful situation.
>
>
>
> Our plan is to rent a nice house somewhere in the UK for a week and gather
> together a few people there to just sit down and slog through docstrings
> for the existing code. The idea is that once the existing code has
> docstrings, the burden of adding the docstrings when adding to or modifying
> petsc4py in the future will be minimal, and the code review process will
> hopefully ensure it happens.
>
>
>
> This doesn’t get us the sort of fully idiomatic Python interface that Jed
> is talking about, but it will produce a big improvement in the usability of
> the Python interface.
>
>
>
> At the moment this is still a ways off and we’re only at the stage that
> we’ve agreed in principle with Lisandro to do it. I think sometime around
> Easter 2023 is a likely timescale.
>

One option that should be considered is an add-on to sowing that generates
the docstring from the C source documentation. It will not work
for everything, but the first 80% removes a lot of busywork, and you can
probably automate a further 10%.

  Thanks,

     Matt


> Cheers,
>
>
>
> David
>
>
>
> *From: *petsc-dev <petsc-dev-bounces at mcs.anl.gov> on behalf of Barry
> Smith <bsmith at petsc.dev>
> *Date: *Thursday, 28 July 2022 at 07:57
> *To: *Jed Brown <jed at jedbrown.org>
> *Cc: *Satish Balay via petsc-dev <petsc-dev at mcs.anl.gov>
> *Subject: *Re: [petsc-dev] PETSc future starting as a new design layer
> that runs on top of PETSc 3?
>
>
>
>
>
> On Jul 27, 2022, at 5:47 PM, Jed Brown <jed at jedbrown.org> wrote:
>
>
>
> PETSc has never used fine grain OO, like per row of a matrix or per
> element in a mesh (compare DMPlex with libMesh).
>
>
>
>   Yes, because even in 1994, we knew that was a dead-end for performance
> and unnecessary as well :-)
>
>
>
> But languages have idioms related to features like iterators, high-order
> functions/closures, traits, error handling/nullability, and multimethods.
> You can usually make a naive interface that gets the job done and allows
> porting applications between languages, but it won't feel as ergonomic or
> compose well with other libraries, nor will it enforce invariants at
> compile time with clear error messages.
>
>
>
> As a result, good language bindings usually involve a more or less
> automatic raw interface that users almost never interact with directly,
> plus an idiomatic layer written by hand. The problem is that development of
> the idiomatic layer (plus documentation and maintenance) requires expertise
> and labor.
>
>
>
>   I doubt we will have the resources with the appropriate expertise to do
> this for more than at most one language. We cannot currently do it with
> Python or Fortran.
>
>
>
> On Wed, Jul 27, 2022, at 2:48 PM, Barry Smith wrote:
>
>
>
>   The API for PETSc is (by design) highly object-oriented; the data
> encapsulation is helpful and rarely gets in the way of performance. The
> object oriented nature makes it easier to be supported by multiple
> languages even when one does not utilize the idiomatic features for
> object-oriented code in Python and Fortran  for example (though one could).
> With GPU's it seems that being heavily object-oriented may be less
> desirable (kernel fusion), MatCOO stuff? I feel we may need to expose more
> array based APIs in the future?
>
>
>
>   It also seems difficult to provide multiple language APIs that properly
> utilize each language's object oriented features automatically from one
> given starting language? Without a lot of thought and effort.
>
>
>
>
>
>
>
> On Jul 27, 2022, at 3:12 PM, Jed Brown <jed at jedbrown.org> wrote:
>
>
>
> I expect PETSc will have some C++ (CUDA/HIP/SYCL) code for the foreseeable
> future, but that doesn't mean the primary implementation language needs to
> be C++, nor that it needs to bleed directly into a public interface. Much
> of the value in PETSc is higher level than GPU numerical kernels.
>
>
>
> One issue is that 1:1 language bindings like we have now for C, Fortran,
> and mostly Python, don't lead to idiomatic code. Idiomatic language
> bindings require thought and documentation.
>
>
>
> On Wed, Jul 27, 2022, at 10:55 AM, Barry Smith wrote:
>
>
>
>   Stan is less than 10 years old, has over 100,000 users and created their
> own language (that gets compiled to C++ code). Their community, like some
> optimization sub-communities) have a history of creating their own
> languages, unlike numerical linear algebra that didn't need to because
> Fortran was perfect for it :-)
>
>
>
>
>
>
>
> On Jul 27, 2022, at 12:29 PM, Justin Chang <jychang48 at gmail.com> wrote:
>
>
>
> FWIW, vendors have poured a lot of effort into making C++ the industry
> standard for HPC, and it will remain that way for a very long time.
> Switching PETSc to a non-C/C++/Python/Fortran language today while still
> enabling GPU/accelerated computing could get ugly from a code
> implementation perspective.
>
>
>
> Not saying you shouldn't consider other languages, but if we want the most
> seamless GPU experience then C++ is still the most pragmatic choice today
> and for the foreseeable future. And it could become even more important on
> tomorrow’s heterogeneous hardware architectures.
>
>
>
> On Tue, Jul 26, 2022 at 11:09 AM Barry Smith <bsmith at petsc.dev> wrote:
>
>
>
>
>
> On Jul 26, 2022, at 11:30 AM, Jed Brown <jed at jedbrown.org> wrote:
>
>
>
> These ownership patterns need to be addressed for reliable interfaces in
> any language, the compiler just forces you to do it (or use the unsafe
> escape hatch) in Rust.
>
>
>
> I think it's necessary in any incremental porting effort for "old" code to
> call "new" code, due to the nature of our composition and callbacks.
>
>
>
>    I would need to see some use cases of this to be convinced.
>
>
>
> On Tue, Jul 26, 2022, at 8:17 AM, Jeremy L Thompson wrote:
>
> I feel like someone has to mention the possibility of Rust.
>
> In libCEED, we've found the FFI to C fairly painless. We made some
> improvements on the core C code of libCEED to facilitate Rust error
> handling and data ownership.
>
> From various prototyping we've done in Jed's group, I think the more
> complex data ownership used in PETSc (as compared to libCEED) is one of the
> more complex issues that would need to be planned out for a Rust focused
> interface.
>
> On 7/25/22 15:34, Barry Smith wrote:
>
>
>
>    A  major problem with writing a completely new version of a large code
> base is that one has to start with nothing and slowly build up to
> everything, which can take years. Years in which you need to continue to
> maintain the old version, people want to continue to add functionality to
> the old version, and people want to continue to use the old version because
> the new version doesn't have "the functionality the user needs" ready yet.
>
>
>
>   Is there an approach where we can have a new PETSc API/language/paradigm
> but start with a very thin layer on the current API so it just works from
> day one?
>
>    - to this would seem to require if PETSc future is not in C, there has
>    to be a very, very easy way and low error-prone way to wrap PETSc current
>    to be called from the new language. For example, how petsc4py wraps seems
>    too manual and too error-prone. C++ can easily and low-error prone call C,
>    any other viable candidates?
>
>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20220728/b87a3313/attachment-0001.html>