[petsc-users] Vec, Mat and binaryfiles.

[Sal Am]

Yes I was just hoping there'd be more than that.

I have tried using one of them as basis:

import PetscBinaryIO
import numpy as np
import scipy.sparse

b_vector = np.array(np.fromfile('Vector_b.bin'
,dtype=np.dtype((np.float64,2))));
A_matrix =
np.array(np.fromfile('Matrix_A_cmplx.bin',dtype=np.dtype((np.float64,2))));
A_indexs = np.array(np.fromfile('Matrix_A_int.bin'
,dtype=np.dtype((np.int32       ,2))));

b_vector = b_vector[:,0] + 1j*b_vector[:,1]
A_matrix = A_matrix[:,0] + 1j*A_matrix[:,1]

A_matrix = scipy.sparse.csr_matrix((A_matrix, (A_indexs[:,0]-1,
A_indexs[:,1]-1)))

vec = b_vector.view(PetscBinaryIO.Vec)

io = PetscBinaryIO.PetscBinaryIO()
io.writeBinaryFile('Vector_b.dat', [vec,])
io.writeMatSciPy('Matrix_A.dat', A_matrix)

I was able to do it convert and read the Vector_b in fine.
So what I am having trouble with is actually converting the
scipy.sparse.csr_matrix into the PETSc format. I thought the writeMatSciPy
would do that by calling writeMatSparse?
Maybe I have misunderstood something here?

All the best,
S

On Thu, Nov 8, 2018 at 10:10 AM Matthew Knepley <knepley at gmail.com> wrote:

> On Thu, Nov 8, 2018 at 4:56 AM Sal Am via petsc-users <
> petsc-users at mcs.anl.gov> wrote:
>
>> Thanks, I missed that,
>>
>> I do care about scalability, the long term plan is to make it fully
>> compatible on a cluster so I guess I would need to convert it to PETSc
>> binary format before reading it in.
>>
>> Is this the file you were referring to
>> https://github.com/erdc/petsc-dev/blob/master/bin/pythonscripts/PetscBinaryIO.py
>>
>> Cannot seem to find docs on how to use it, do you happen to have any
>> examples?
>>
>
> The documentation is at the top of the file. You can see it by clicking
> the link.
>
>    Matt
>
>
>> All the best,
>> S
>>
>>
>>
>>
>> On Wed, Nov 7, 2018 at 4:07 PM Jed Brown <jed at jedbrown.org> wrote:
>>
>>> Please always use "reply-all" so that your messages go to the list.
>>> This is standard mailing list etiquette.  It is important to preserve
>>> threading for people who find this discussion later and so that we do
>>> not waste our time re-answering the same questions that have already
>>> been answered in private side-conversations.  You'll likely get an
>>> answer faster that way too.
>>>
>>> Sal Am <tempohoper at gmail.com> writes:
>>>
>>> > Thank you Jed for the quick response!
>>> >
>>> >>
>>> >> Yes, of course the formats would have to match.  I would recommend
>>> >> writing the files in an existing format such as PETSc's binary format.
>>> >>
>>> >
>>> > Unfortunately I do not think I can change the source code to output
>>> those
>>> > two files in PETSc format and it would probably take a very long time
>>> > converting everything into PETSc (it is not even my code).
>>>
>>> File-based workflows are very often bottlenecks.  You can also use any
>>> convenient software (e.g., Python or Matlab/Octave) to convert your
>>> custom binary formats to PETSc binary format (see PetscBinaryIO provided
>>> with PETSc), at which point you'll be able to read in parallel.  If you
>>> don't care about scalability or only need to read once, then you can
>>> write code of the type you propose.
>>>
>>> > Do you have any other suggestions on how to read in those two complex
>>> > binary files? Also why would it be more difficult to parallelise as
>>> > I thought getting the two files in PETSc vector format would allow me
>>> to
>>> > use the rest of PETSc library the usual way?
>>> >
>>> > Kind regards,
>>> > S
>>> >
>>> >
>>> > On Mon, Nov 5, 2018 at 4:49 PM Jed Brown <jed at jedbrown.org> wrote:
>>> >
>>> >> Sal Am via petsc-users <petsc-users at mcs.anl.gov> writes:
>>> >>
>>> >> > Hi,
>>> >> >
>>> >> > I am trying to solve a Ax=b complex system. the vector b and
>>> "matrix" A
>>> >> are
>>> >> > both binary and NOT created by PETSc. So I keep getting error
>>> messages
>>> >> that
>>> >> > they are not correct format when I read the files with
>>> >> PetscViewBinaryOpen,
>>> >> > after some digging it seems that one cannot just read a binary file
>>> that
>>> >> > was created by another software.
>>> >>
>>> >> Yes, of course the formats would have to match.  I would recommend
>>> >> writing the files in an existing format such as PETSc's binary format.
>>> >> While the method you describe can be made to work, it will be more
>>> work
>>> >> to make it parallel.
>>> >>
>>> >> > How would I go on to solve this problem?
>>> >> >
>>> >> > More info and trials:
>>> >> >
>>> >> > "matrix" A consists of two files, one that contains row column index
>>> >> > numbers and one that contains the non-zero values. So what I would
>>> have
>>> >> to
>>> >> > do is multiply the last term in a+b with PETSC_i to get a real +
>>> >> imaginary
>>> >> > vector A.
>>> >> >
>>> >> > vector b is in binary, so what I have done so far (not sure if it
>>> works)
>>> >> is:
>>> >> >
>>> >> > std::ifstream input("Vector_b.bin", std::ios::binary );
>>> >> > while (input.read(reinterpret_cast<char*>(&v), sizeof(float)))
>>> >> >      ierr    =
>>> VecSetValues(u,1,&iglobal,&v,INSERT_VALUES);CHKERRQ(ierr);
>>> >> >
>>> >> > where v is a PetscScalar.
>>> >> >
>>> >> > Once I am able to read both matrices I think I can figure out the
>>> solvers
>>> >> > to solve the system.
>>> >> >
>>> >> > All the best,
>>> >> > S
>>> >>
>>>
>>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
> <http://www.cse.buffalo.edu/~knepley/>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20181108/65c30114/attachment.html>