<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:0 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:10.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
p.qt-qt-, li.qt-qt-, div.qt-qt-
{mso-style-name:qt-qt-;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:809595255;
mso-list-template-ids:1648633816;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:\F0B7 ;
mso-level-tab-stop:36.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:72.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:"Courier New";
mso-bidi-font-family:"Times New Roman";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7 ;
mso-level-tab-stop:108.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:\F0A7 ;
mso-level-tab-stop:144.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:\F0A7 ;
mso-level-tab-stop:180.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:\F0A7 ;
mso-level-tab-stop:216.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:\F0A7 ;
mso-level-tab-stop:252.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:\F0A7 ;
mso-level-tab-stop:288.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:\F0A7 ;
mso-level-tab-stop:324.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style>
</head>
<body lang="EN-GB" link="blue" vlink="purple" style="word-wrap:break-word;-webkit-nbsp-mode: space;line-break:after-white-space">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">This seems like an opportune moment to flag that we’re proposing a sprint at some point in 2023 to document petsc4py. Right now there are no docstrings in petsc4py, which means
users have to infer what the corresponding C function name is and read the docstring for that, then understand enough about how the idiom changes between the C and Python interfaces to re-interpret the C documentation to Python. This is a pretty awful situation.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Our plan is to rent a nice house somewhere in the UK for a week and gather together a few people there to just sit down and slog through docstrings for the existing code. The idea
is that once the existing code has docstrings, the burden of adding the docstrings when adding to or modifying petsc4py in the future will be minimal, and the code review process will hopefully ensure it happens.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">This doesn’t get us the sort of fully idiomatic Python interface that Jed is talking about, but it will produce a big improvement in the usability of the Python interface.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">At the moment this is still a ways off and we’re only at the stage that we’ve agreed in principle with Lisandro to do it. I think sometime around Easter 2023 is a likely timescale.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Cheers,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">David<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">petsc-dev <petsc-dev-bounces@mcs.anl.gov> on behalf of Barry Smith <bsmith@petsc.dev><br>
<b>Date: </b>Thursday, 28 July 2022 at 07:57<br>
<b>To: </b>Jed Brown <jed@jedbrown.org><br>
<b>Cc: </b>Satish Balay via petsc-dev <petsc-dev@mcs.anl.gov><br>
<b>Subject: </b>Re: [petsc-dev] PETSc future starting as a new design layer that runs on top of PETSc 3?<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt"><br>
<br>
<o:p></o:p></span></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt">On Jul 27, 2022, at 5:47 PM, Jed Brown <<a href="mailto:jed@jedbrown.org">jed@jedbrown.org</a>> wrote:<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica">PETSc has never used fine grain OO, like per row of a matrix or per element in a mesh (compare DMPlex with libMesh).
<o:p></o:p></span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt"> Yes, because even in 1994, we knew that was a dead-end for performance and unnecessary as well :-)<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt"><br>
<br>
<o:p></o:p></span></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica">But languages have idioms related to features like iterators, high-order functions/closures, traits, error handling/nullability, and multimethods. You can usually make a naive interface
that gets the job done and allows porting applications between languages, but it won't feel as ergonomic or compose well with other libraries, nor will it enforce invariants at compile time with clear error messages.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica">As a result, good language bindings usually involve a more or less automatic raw interface that users almost never interact with directly, plus an idiomatic layer written by hand. The
problem is that development of the idiomatic layer (plus documentation and maintenance) requires expertise and labor.<o:p></o:p></span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt"> I doubt we will have the resources with the appropriate expertise to do this for more than at most one language. We cannot currently do it with Python or Fortran.<br>
<br>
<o:p></o:p></span></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica">On Wed, Jul 27, 2022, at 2:48 PM, Barry Smith wrote:<o:p></o:p></span></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt;font-variant-caps: normal;orphans: auto;text-align:start;widows: auto;-webkit-text-size-adjust: auto;-webkit-text-stroke-width: 0px;overflow-wrap: break-word;word-spacing:0px" id="qt">
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"> The API for PETSc is (by design) highly object-oriented; the data encapsulation is helpful and rarely gets in the way of performance. The object oriented nature makes it easier to be
supported by multiple languages even when one does not utilize the idiomatic features for object-oriented code in Python and Fortran for example (though one could). With GPU's it seems that being heavily object-oriented may be less desirable (kernel fusion),
MatCOO stuff? I feel we may need to expose more array based APIs in the future?<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"> It also seems difficult to provide multiple language APIs that properly utilize each language's object oriented features automatically from one given starting language? Without a lot
of thought and effort.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"><o:p> </o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"><o:p> </o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"><o:p> </o:p></span></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica">On Jul 27, 2022, at 3:12 PM, Jed Brown <<a href="mailto:jed@jedbrown.org">jed@jedbrown.org</a>> wrote:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"><o:p> </o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica">I expect PETSc will have some C++ (CUDA/HIP/SYCL) code for the foreseeable future, but that doesn't mean the primary implementation language needs to be C++, nor that it needs to bleed
directly into a public interface. Much of the value in PETSc is higher level than GPU numerical kernels.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica">One issue is that 1:1 language bindings like we have now for C, Fortran, and mostly Python, don't lead to idiomatic code. Idiomatic language bindings require thought and documentation. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica">On Wed, Jul 27, 2022, at 10:55 AM, Barry Smith wrote:<o:p></o:p></span></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt;font-variant-caps: normal;text-align:start;-webkit-text-stroke-width: 0px;text-decoration-line: none;text-decoration-thickness: initial;overflow-wrap: break-word;word-spacing:0px" id="qt-qt">
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"> Stan is less than 10 years old, has over 100,000 users and created their own language (that gets compiled to C++ code). Their community, like some optimization sub-communities) have
a history of creating their own languages, unlike numerical linear algebra that didn't need to because Fortran was perfect for it :-)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"><o:p> </o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"><o:p> </o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"><o:p> </o:p></span></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica">On Jul 27, 2022, at 12:29 PM, Justin Chang <<a href="mailto:jychang48@gmail.com">jychang48@gmail.com</a>> wrote:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"><o:p> </o:p></span></p>
</div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica">FWIW, vendors have poured a lot of effort into making C++ the industry standard for HPC, and it will remain that way for a very long time. Switching PETSc to a non-C/C++/Python/Fortran
language today while still enabling GPU/accelerated computing could get ugly from a code implementation perspective. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica">Not saying you shouldn't consider other languages, but if we want the most seamless GPU experience then C++ is still the most pragmatic choice today and for the foreseeable future. And
it could become even more important on tomorrow’s heterogeneous hardware architectures.<o:p></o:p></span></p>
</div>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"><o:p> </o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica">On Tue, Jul 26, 2022 at 11:09 AM Barry Smith <<a href="mailto:bsmith@petsc.dev" target="_blank">bsmith@petsc.dev</a>> wrote:<o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"><o:p> </o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"><o:p> </o:p></span></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica">On Jul 26, 2022, at 11:30 AM, Jed Brown <<a href="mailto:jed@jedbrown.org" target="_blank">jed@jedbrown.org</a>> wrote:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"><o:p> </o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica">These ownership patterns need to be addressed for reliable interfaces in any language, the compiler just forces you to do it (or use the unsafe escape hatch) in Rust.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica">I think it's necessary in any incremental porting effort for "old" code to call "new" code, due to the nature of our composition and callbacks.<o:p></o:p></span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"> I would need to see some use cases of this to be convinced. <o:p></o:p></span></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica">On Tue, Jul 26, 2022, at 8:17 AM, Jeremy L Thompson wrote:<o:p></o:p></span></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt;font-variant-caps: normal;text-align:start;text-decoration-line: none;text-decoration-thickness: initial;word-spacing:0px" id="qt-qt-m_2446162622667342447gmail-m_787130091436462510qt">
<p class="qt-qt-"><span style="font-size:13.5pt;font-family:Helvetica">I feel like someone has to mention the possibility of Rust.<o:p></o:p></span></p>
<p class="qt-qt-"><span style="font-size:13.5pt;font-family:Helvetica">In libCEED, we've found the FFI to C fairly painless. We made some improvements on the core C code of libCEED to facilitate Rust error handling and data ownership.<o:p></o:p></span></p>
<p class="qt-qt-"><span style="font-size:13.5pt;font-family:Helvetica">>From various prototyping we've done in Jed's group, I think the more complex data ownership used in PETSc (as compared to libCEED) is one of the more complex issues that would need to be
planned out for a Rust focused interface.<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica">On 7/25/22 15:34, Barry Smith wrote:<o:p></o:p></span></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"> A major problem with writing a completely new version of a large code base is that one has to start with nothing and slowly build up to everything, which can take years. Years in
which you need to continue to maintain the old version, people want to continue to add functionality to the old version, and people want to continue to use the old version because the new version doesn't have "the functionality the user needs" ready yet.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Helvetica"> Is there an approach where we can have a new PETSc API/language/paradigm but start with a very thin layer on the current API so it just works from day one?<o:p></o:p></span></p>
</div>
<div>
<ul type="disc">
<li class="qt-qt-" style="mso-list:l0 level1 lfo1"><span style="font-size:13.5pt;font-family:Helvetica">to this would seem to require if PETSc future is not in C, there has to be a very, very easy way and low error-prone way to wrap PETSc current to be called
from the new language. For example, how petsc4py wraps seems too manual and too error-prone. C++ can easily and low-error prone call C, any other viable candidates?<o:p></o:p></span></li></ul>
</div>
</blockquote>
</blockquote>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
</blockquote>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
</body>
</html>