<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Jul 27, 2022, at 5:47 PM, Jed Brown <<a href="mailto:jed@jedbrown.org" class="">jed@jedbrown.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta charset="UTF-8" class=""><div style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">PETSc has never used fine grain OO, like per row of a matrix or per element in a mesh (compare DMPlex with libMesh). </div></div></blockquote><div><br class=""></div><div> Yes, because even in 1994, we knew that was a dead-end for performance and unnecessary as well :-)</div><br class=""><blockquote type="cite" class=""><div class=""><div style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">But languages have idioms related to features like iterators, high-order functions/closures, traits, error handling/nullability, and multimethods. You can usually make a naive interface that gets the job done and allows porting applications between languages, but it won't feel as ergonomic or compose well with other libraries, nor will it enforce invariants at compile time with clear error messages.<br class=""></div><div style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br class=""></div><div style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">As a result, good language bindings usually involve a more or less automatic raw interface that users almost never interact with directly, plus an idiomatic layer written by hand. The problem is that development of the idiomatic layer (plus documentation and maintenance) requires expertise and labor.<br class=""></div></div></blockquote><div><br class=""></div> I doubt we will have the resources with the appropriate expertise to do this for more than at most one language. We cannot currently do it with Python or Fortran.<br class=""><blockquote type="cite" class=""><div class=""><div style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br class=""></div><div style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">On Wed, Jul 27, 2022, at 2:48 PM, Barry Smith wrote:<br class=""></div><blockquote type="cite" id="qt" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; overflow-wrap: break-word;" class=""><div class="qt-"><br class=""></div><div class=""> The API for PETSc is (by design) highly object-oriented; the data encapsulation is helpful and rarely gets in the way of performance. The object oriented nature makes it easier to be supported by multiple languages even when one does not utilize the idiomatic features for object-oriented code in Python and Fortran for example (though one could). With GPU's it seems that being heavily object-oriented may be less desirable (kernel fusion), MatCOO stuff? I feel we may need to expose more array based APIs in the future?<br class=""></div><div class="qt-"><br class=""></div><div class="qt-"> It also seems difficult to provide multiple language APIs that properly utilize each language's object oriented features automatically from one given starting language? Without a lot of thought and effort.<br class=""></div><div class="qt-"><br class=""></div><div class="qt-"><div class=""><br class=""></div><div class=""><div class=""><br class=""></div><blockquote type="cite" class="qt-"><div class="qt-">On Jul 27, 2022, at 3:12 PM, Jed Brown <<a href="mailto:jed@jedbrown.org" class="qt-">jed@jedbrown.org</a>> wrote:<br class=""></div><div class=""><br class=""></div><div class="qt-"><div class="qt-" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: initial;">I expect PETSc will have some C++ (CUDA/HIP/SYCL) code for the foreseeable future, but that doesn't mean the primary implementation language needs to be C++, nor that it needs to bleed directly into a public interface. Much of the value in PETSc is higher level than GPU numerical kernels.<br class=""></div><div class="qt-" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: initial;"><br class=""></div><div class="qt-" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: initial;">One issue is that 1:1 language bindings like we have now for C, Fortran, and mostly Python, don't lead to idiomatic code. Idiomatic language bindings require thought and documentation. <br class=""></div><div class="qt-" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: initial;"><br class=""></div><div class="qt-" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: initial;">On Wed, Jul 27, 2022, at 10:55 AM, Barry Smith wrote:<br class=""></div><blockquote type="cite" id="qt-qt" class="qt-" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: initial; overflow-wrap: break-word;"><div class="qt-qt-"><br class=""></div><div class="qt-"> Stan is less than 10 years old, has over 100,000 users and created their own language (that gets compiled to C++ code). Their community, like some optimization sub-communities) have a history of creating their own languages, unlike numerical linear algebra that didn't need to because Fortran was perfect for it :-)<br class=""></div><div class="qt-qt-"><br class=""></div><div class="qt-qt-"><div class="qt-"><br class=""></div><div class="qt-"><div class="qt-"><br class=""></div><blockquote type="cite" class="qt-qt-"><div class="qt-qt-">On Jul 27, 2022, at 12:29 PM, Justin Chang <<a href="mailto:jychang48@gmail.com" class="qt-qt-">jychang48@gmail.com</a>> wrote:<br class=""></div><div class="qt-"><br class=""></div><div class="qt-qt-"><div class="qt-qt-"><div dir="ltr" class="qt-qt-"><div class="qt-">FWIW, vendors have poured a lot of effort into making C++ the industry standard for HPC, and it will remain that way for a very long time. Switching PETSc to a non-C/C++/Python/Fortran language today while still enabling GPU/accelerated computing could get ugly from a code implementation perspective. <br class=""></div><div class="qt-"><br class=""></div><div class="qt-">Not saying you shouldn't consider other languages, but if we want the most seamless GPU experience then C++ is still the most pragmatic choice today and for the foreseeable future. And it could become even more important on tomorrow’s heterogeneous hardware architectures.<br class=""></div></div></div><div class="qt-qt-"><div class="qt-"><br class=""></div><div class="qt-qt-gmail_quote"><div dir="ltr" class="qt-qt-gmail_attr">On Tue, Jul 26, 2022 at 11:09 AM Barry Smith <<a href="mailto:bsmith@petsc.dev" target="_blank" class="qt-qt-">bsmith@petsc.dev</a>> wrote:<br class=""></div><blockquote class="qt-qt-gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; padding-left: 1ex; border-left-color: rgb(204, 204, 204);"><div class="qt-qt-"><div class="qt-"><br class=""></div><div class="qt-qt-"><div class="qt-"><br class=""></div><blockquote type="cite" class="qt-qt-"><div class="qt-qt-">On Jul 26, 2022, at 11:30 AM, Jed Brown <<a href="mailto:jed@jedbrown.org" target="_blank" class="qt-qt-">jed@jedbrown.org</a>> wrote:<br class=""></div><div class="qt-"><br class=""></div><div class="qt-qt-"><div class="qt-qt-" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration-line: none; text-decoration-thickness: initial;">These ownership patterns need to be addressed for reliable interfaces in any language, the compiler just forces you to do it (or use the unsafe escape hatch) in Rust.<br class=""></div><div class="qt-qt-" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration-line: none; text-decoration-thickness: initial;"><br class=""></div><div class="qt-qt-" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration-line: none; text-decoration-thickness: initial;">I think it's necessary in any incremental porting effort for "old" code to call "new" code, due to the nature of our composition and callbacks.<br class=""></div></div></blockquote><div class="qt-qt-"><br class=""></div><div class="qt-"> I would need to see some use cases of this to be convinced. <br class=""></div><blockquote type="cite" class="qt-qt-"><div class="qt-qt-"><div class="qt-qt-" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration-line: none; text-decoration-thickness: initial;"><br class=""></div><div class="qt-qt-" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration-line: none; text-decoration-thickness: initial;">On Tue, Jul 26, 2022, at 8:17 AM, Jeremy L Thompson wrote:<br class=""></div><blockquote type="cite" id="qt-qt-m_2446162622667342447gmail-m_787130091436462510qt" class="qt-qt-" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration-line: none; text-decoration-thickness: initial;"><p class="qt-qt-" style="font-family: Helvetica;">I feel like someone has to mention the possibility of Rust.<br class=""></p><p class="qt-qt-" style="font-family: Helvetica;">In libCEED, we've found the FFI to C fairly painless. We made some improvements on the core C code of libCEED to facilitate Rust error handling and data ownership.<br class=""></p><p class="qt-qt-" style="font-family: Helvetica;">From various prototyping we've done in Jed's group, I think the more complex data ownership used in PETSc (as compared to libCEED) is one of the more complex issues that would need to be planned out for a Rust focused interface.<br class=""></p><div class="qt-qt-" style="font-family: Helvetica;">On 7/25/22 15:34, Barry Smith wrote:<br class=""></div><blockquote type="cite" class="qt-qt-" style="font-family: Helvetica;"><div class="qt-qt-" style="font-family: Helvetica;"><br class=""></div><div class="qt-qt-" style="font-family: Helvetica;"> A major problem with writing a completely new version of a large code base is that one has to start with nothing and slowly build up to everything, which can take years. Years in which you need to continue to maintain the old version, people want to continue to add functionality to the old version, and people want to continue to use the old version because the new version doesn't have "the functionality the user needs" ready yet.<br class=""></div><div class="qt-qt-" style="font-family: Helvetica;"><br class=""></div><div class="qt-qt-" style="font-family: Helvetica;"> Is there an approach where we can have a new PETSc API/language/paradigm but start with a very thin layer on the current API so it just works from day one?<br class=""></div><div class="qt-qt-" style="font-family: Helvetica;"><ul class="qt-qt-" style="font-family: Helvetica;"><li class="qt-qt-" style="font-family: Helvetica;">to this would seem to require if PETSc future is not in C, there has to be a very, very easy way and low error-prone way to wrap PETSc current to be called from the new language. For example, how petsc4py wraps seems too manual and too error-prone. C++ can easily and low-error prone call C, any other viable candidates?</li></ul></div></blockquote></blockquote></div></blockquote></div></div></blockquote></div></div></div></blockquote></div></div></blockquote></div></blockquote></div></div></blockquote></div></blockquote></div><br class=""></body></html>