[petsc-users] PETSc read binary matrix row by row

[Barry Smith]

   Take a look at MatLoad_SeqAIJ_Binary(). You will see how it is easy to simplify that code to get what you need.  You can call PetscViewerBinaryGetDescriptor() and then use lseek() to skip over all the integer part of the matrix storage (the row lengths and column indices) and not even read that into memory at all.

  Since the file stores all the numerical values after the integer part you won't need to read a row at a time, just loop over "big chunks" and read a big chunk at a time of the numerical part and add that big chunk to the appropriate place in the in-memory AIJ matrix.  You are free to pick any big chunk size, it should be at least 1000s of doubles for good performance.

   Barry


> On May 22, 2025, at 2:23 AM, Donald Duck <superdduck88 at gmail.com> wrote:
> 
> Hello everyone
> 
> A piece of c++ code writes PETSc AIJ matrices to binary files. My task is to compute an average matrix of these AIJ matrices. Therefore I read the first matrix with petsc4py and then start to add the other matrices to it. All matrixes always have the same size, shape, nnz etc.
> 
> However in some cases these matrices are large and only one of it fits into memory and the reading/writing takes a significant amout of time. Is there a way to read it row by row to prevent memory overflow?
> 
> I'm also open for other suggestions, thanks in advance!
> 
> Raphael

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20250522/b02c81d4/attachment.html>