[petsc-users] Question about petsc4py createWithArray function

Wed May 8 02:53:49 CDT 2024

Hi Stefano,

Thank you for this. Your example was invaluable in helping me figure out what is going on. (I did not know about about sys.getrefcount - I’m only just starting to learn python.) It was indeed the refcount. Until the Vec is destroyed it can’t go to zero and my array destructor method won’t be triggered (which is not really a problem because the memory is freed elsewhere). I take back what I said about there being a memory leak in petsc4py. Sorry.

Thanks again for your help!

Best,

Samar

On May 6, 2024, at 2:09 PM, Stefano Zampini <stefano.zampini at gmail.com> wrote:

Samar

After a second look, I believe the petsc4py code is correct. You can test it using the script below.
After destroy is called (or del), the reference count of the numpy array is back to its initial state.
Maybe you are not calling del or destroy? a MWE would help to understand your use case

kl-18448:~ szampini$ cat t.py
import sys
import numpy as np
from petsc4py import PETSc

x = np.zeros(4, dtype=PETSc.ScalarType)
print('initial ref count',sys.getrefcount(x))

v = PETSc.Vec().createWithArray(x)
print('after create',sys.getrefcount(x))

# check if they share memory
v.view()
x[1] = 2
v.view()

# free
v.destroy()
# you can also call del
# del v
print('after destroy',sys.getrefcount(x))

kl-18448:~ szampini$ python t.py
initial ref count 2
after create 3
Vec Object: 1 MPI process
  type: seq
0.
0.
0.
0.
Vec Object: 1 MPI process
  type: seq
0.
2.
0.
0.
after destroy 2

Il giorno lun 6 mag 2024 alle ore 08:49 Samar Khatiwala <samar.khatiwala at earth.ox.ac.uk<mailto:samar.khatiwala at earth.ox.ac.uk>> ha scritto:
Hi Stefano,

Thanks for looking into this. Since createWithArray calls VecCreateMPIWithArray which, as Matt noted and is documented (https://urldefense.us/v3/__https://petsc.org/main/manualpages/Vec/VecCreateMPIWithArray/__;!!G_uCfscf7eWS!fl-7I7xkuEmTFH8M9EaARIZL5vhGFWkIk5bVIGt0bkt_cYcuwPSwgOWaec1-DvYwiKWxA5q0HS0VTLBmD4NnSe1PfG057Y4SBd-8qs8$ ) doesn’t free the memory, then there’s a memory leak (and, furthermore, calling del on the original array will have no effect).

Lisandro: would be great if you can provide some guidance.

Thanks,

Samar

On May 3, 2024, at 12:45 PM, Stefano Zampini <stefano.zampini at gmail.com<mailto:stefano.zampini at gmail.com>> wrote:

While waiting for our Python wizard to shed light on this, I note that, from the documentation of PyArray_FROM_OTF https://urldefense.us/v3/__https://numpy.org/devdocs/user/c-info.how-to-extend.html*converting-an-arbitrary-sequence-object__;Iw!!G_uCfscf7eWS!fl-7I7xkuEmTFH8M9EaARIZL5vhGFWkIk5bVIGt0bkt_cYcuwPSwgOWaec1-DvYwiKWxA5q0HS0VTLBmD4NnSe1PfG057Y4S1ro4qy0$ , we have

The object can be any Python object convertible to an ndarray. If the object is already (a subclass of) the ndarray that satisfies the requirements then a new reference is returned.

I guess we should call "del" on the ndarray returned by iarray_s after having called  self.set_attr('__array__', array) in this case, but let's wait for Lisandro to confirm

Il giorno ven 3 mag 2024 alle ore 11:42 Samar Khatiwala <samar.khatiwala at earth.ox.ac.uk<mailto:samar.khatiwala at earth.ox.ac.uk>> ha scritto:
This Message Is From an External Sender
This message came from outside your organization.

Hi Matt,

Thanks so much for the quick reply!

Regarding #2, I put some debug statement in my code and what I find is that when I use createWithArray on my Cython-allocated numpy array, the destructor I set for it is no longer called when I delete the array. (If I don’t use createWithArray then the destructor is triggered.) I interpret that to suggest that the petsc4py Vec is somehow ’taking over’ management of the numpy array. But I don’t understand where that could be happening. (I don’t think it has to do with the actual freeing of memory by PETSc's VecDestroy.)

createWithArray calls iarray_s which in turn calls PyArray_FROM_OTF. Could it be there’s something going on there? The numpy documentation is unclear.

Lisandro: do you have any thoughts on this?

Thanks,

Samar

On May 2, 2024, at 11:56 PM, Matthew Knepley <knepley at gmail.com<mailto:knepley at gmail.com>> wrote:

On Thu, May 2, 2024 at 12:53 PM Samar Khatiwala <samar.khatiwala at earth.ox.ac.uk<mailto:samar.khatiwala at earth.ox.ac.uk>> wrote:
This Message Is From an External Sender
This message came from outside your organization.

Hello,

I have a couple of questions about createWithArray in petsc4py:

1) What is the correct usage for creating a standard MPI Vec with it? Something like this seems to work but is it right?:

On each rank do:
a = np.zeros(localSize)
v = PETSc.Vec().createWithArray(a, comm=PETSc.COMM_WORLD)

Is that all it takes?

That looks right to me.

2) Who ‘owns’ the underlying memory for a Vec created with the createWithArray method, i.e., who is responsible for managing it and doing garbage collection? In my problem, the numpy array is created in a Cython module where memory is allocated, and a pointer to it is associated with a numpy ndarray via PyArray_SimpleNewFromData and PyArray_SetBaseObject. I have a deallocator method of my own that is called when the numpy array is deleted/goes out of scope/whenever python does garbage collection. All of that works fine. But if I use this array to create a Vec with createWithArray what happens when the Vec is, e.g., destroyed? Will my deallocator be called?

No. The PETSc struct will be deallocated, but the storage will not be touched.

  Thanks,

     Matt

Or does petsc4py know that it doesn’t own the memory and won’t attempt to free it? I can’t quite figure out from the petsc4py code what is going on. And help would be appreciated.

Thanks very much.

Samar

--
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener

https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!fl-7I7xkuEmTFH8M9EaARIZL5vhGFWkIk5bVIGt0bkt_cYcuwPSwgOWaec1-DvYwiKWxA5q0HS0VTLBmD4NnSe1PfG057Y4SmVADDr8$ <https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!dXZuKR18IO2hZ4uPEqHaG0Jb0ALPgw1T0dm4HAWfyBfsCvkYTmq7aG7yw3yc-5CVhcFMaUL3WzGM8YHlBTTA_JMs_dYUCTX81-bKyrM$>

--
Stefano

--
Stefano

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20240508/46636932/attachment-0001.html>