[petsc-users] PetscPythonInitialize, KSPPYTHON, PCPYTHON, etc

Fri Mar 14 12:09:30 CDT 2025

On Fri, Mar 14, 2025 at 12:53 PM Eric Chamberland <
Eric.Chamberland at giref.ulaval.ca> wrote:

> Thanks Matthew!
>
> I hope you're doing well.
>
>
Hi! I know there is still a Plex bug submitted, but this is the busiest
time of the year. I will fix it soon.

> I have a question regarding a change I did on our code: I modified the
> code to call PetscPythonInitialize(PETSC_NULLPTR, PETSC_NULLPTR) after
> SlepcInitialize.
>
> Our CI ran successfully across 20 different environments (including
> Valgrind), but we encountered a segmentation fault on Ubuntu 24.04 when
> calling PetscPythonInitialize within our Python module (built using
> Pybind11 bindings).
>
> On Ubuntu, calling PetscPythonInitialize from within our Python 3 module
> results in a segfault with the following backtrace:
>
> This is weird. When I have seen things like this before, it is because
someone is not dynamically linking properly, so that
static variables are duplicated, and they get out of sync.

This is why I always use  -python to have PETSc do that initialization
automatically, and it guaranteed to be linked to the same
libpython that the SNESPYTHON is using.

  Thanks,

     Matt

> Thread 1 "python3" received signal SIGSEGV, Segmentation fault.
> 0x00001555339b372a in PyList_New () from
> /lib/x86_64-linux-gnu/libpython3.12.so.1.0
> (gdb) bt
> #0  0x00001555339b372a in PyList_New () from
> /lib/x86_64-linux-gnu/libpython3.12.so.1.0
> #1  0x0000155533ad89d7 in PyImport_Import () from
> /lib/x86_64-linux-gnu/libpython3.12.so.1.0
> #2  0x0000155533ad8c80 in PyImport_ImportModule () from
> /lib/x86_64-linux-gnu/libpython3.12.so.1.0
> #3  0x000015553f437c0c in PetscPythonInitialize (pyexe=0x0, pylib=0x0) at
> /tmp/build_openmpi-4.1.6-opt/petsc-3.21.6-debug/src/sys/python/pythonsys.c
> :242
>
> I am puzzled by this segfault...
>
> Some relevant details:
>
> grep -r PETSC_PYTHON_EXE /opt/petsc-3.21.6_debug_openmpi-4.1.6/include/
> /opt/petsc-3.21.6_debug_openmpi-4.1.6/include/petscconf.h:#define
> PETSC_PYTHON_EXE "/usr/bin/python3"
>
> ls -la /lib/x86_64-linux-gnu/libpython3.12*
> lrwxrwxrwx 1 root root       58 Feb  4 09:48
> /lib/x86_64-linux-gnu/libpython3.12.a ->
> ../python3.12/config-3.12-x86_64-linux-gnu/libpython3.12.a
> lrwxrwxrwx 1 root root       60 Feb  4 09:48
> /lib/x86_64-linux-gnu/libpython3.12d.a ->
> ../python3.12/config-3.12d-x86_64-linux-gnu/libpython3.12d.a
> lrwxrwxrwx 1 root root       19 Feb  4 09:48 /lib/x86_64-linux-gnu/
> libpython3.12d.so -> libpython3.12d.so.1
> lrwxrwxrwx 1 root root       21 Feb  4 09:48
> /lib/x86_64-linux-gnu/libpython3.12d.so.1 -> libpython3.12d.so.1.0
> -rw-r--r-- 1 root root 34018416 Feb  4 09:48
> /lib/x86_64-linux-gnu/libpython3.12d.so.1.0
> lrwxrwxrwx 1 root root       18 Feb  4 09:48 /lib/x86_64-linux-gnu/
> libpython3.12.so -> libpython3.12.so.1
> lrwxrwxrwx 1 root root       20 Feb  4 09:48
> /lib/x86_64-linux-gnu/libpython3.12.so.1 -> libpython3.12.so.1.0
> -rw-r--r-- 1 root root  9055112 Feb  4 09:48
> /lib/x86_64-linux-gnu/libpython3.12.so.1.0
>
> (I installed the debug ("d") versions to get debug symbols, but the
> segfault occurs even without them.)
>
> Interestingly, on the same Ubuntu setup, calling PetscPythonInitialize
> within our *pure C++* code works fine, and the "pc_type python" example
> runs successfully.
>
> This issue only occurs when calling PetscPythonInitialize from our *Python
> module*.
>
> Any ideas?
>
> Could this be something that has already been fixed in PETSc 3.22.x?
>
> Thanks,
>
> Eric
>
>
> On 2025-03-12 16:48, Matthew Knepley wrote:
>
> On Wed, Mar 12, 2025 at 4:34 PM Eric Chamberland via petsc-users <
> petsc-users at mcs.anl.gov> wrote:
>
>> Hi,
>>
>> just a naive question: looking at KSPPYTHON and PCPYTHON, we saw that
>> there is only 1 example available.
>>
>> We are asking ourself: is it still supported and can we start
>> developping ou PCs and KSPs on top of it?
>>
>> Or is there a "new" replacement for these?
>>
>
> I think the reason that there are so few examples is that many examples
> exist in other packages, such as Firedrake, and they are the main
> consumers. KSPPYTHON is a way to write KSPSHELL using Python rather than C,
> and we mostly write C.
>
> I will say that recently we fixed everything so that PETSc errors and
> Python exceptions are passed correctly up the stack, and debugging these
> things should be easy. I have been debugging the PyVista visualization, and
> I can change the Python in one window and run in the other. It is easy.
>
>   Thanks,
>
>      Matt
>
>
>> Thanks,
>>
>> Eric
>>
>> --
>> Eric Chamberland, ing., M. Ing
>> Professionnel de recherche
>> GIREF/Université Laval
>> (418) 656-2131 poste 41 22 42
>>
>>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!dXr7aqf6xruDXqpMNsJ0FcuY0Tm7N9gm65siheq7oGM02T9B8cFX_qKELOCxmaaUJHR3lkefANYmVinLW3QQ$ 
> <https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!dXr7aqf6xruDXqpMNsJ0FcuY0Tm7N9gm65siheq7oGM02T9B8cFX_qKELOCxmaaUJHR3lkefANYmVrRcOsA9$ >
>
> --
> Eric Chamberland, ing., M. Ing
> Professionnel de recherche
> GIREF/Université Laval
> (418) 656-2131 poste 41 22 42
>
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!dXr7aqf6xruDXqpMNsJ0FcuY0Tm7N9gm65siheq7oGM02T9B8cFX_qKELOCxmaaUJHR3lkefANYmVinLW3QQ$  <https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!dXr7aqf6xruDXqpMNsJ0FcuY0Tm7N9gm65siheq7oGM02T9B8cFX_qKELOCxmaaUJHR3lkefANYmVrRcOsA9$ >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20250314/72902578/attachment-0001.html>