<div dir="ltr"><div dir="ltr"><div>For anyone in the future, to read in binary files written using raw C++ write and to convert to PETSc format using the PetscBinaryIO.py script:</div><div>"""</div><div>Ax = b</div><div>Where mat A is divided into A index and A non-zero elements. <br></div><div>"""<br></div><div>import PetscBinaryIO<br>import numpy as np<br>import scipy.sparse<br></div><div><br></div><div>b_vector = np.array(np.fromfile('Vector_b.bin' ,dtype=np.dtype((np.float64,2))));<br>A_matrix = np.array(np.fromfile('Matrix_A_cmplx.bin',dtype=np.dtype((np.float64,2)))); # non-zero elements.<br>A_indexs = np.array(np.fromfile('Matrix_A_int.bin' ,dtype=np.dtype((np.int32 ,2)))); # row column indexes for the non zero elements<br><br>b_vector = b_vector[:,0] + 1j*b_vector[:,1]<br>A_matrix = A_matrix[:,0] + 1j*A_matrix[:,1]<br><br>A_matrix = scipy.sparse.csr_matrix((A_matrix, (A_indexs[:,0]-1, A_indexs[:,1]-1))) <br><br>vec = b_vector.view(PetscBinaryIO.Vec)<br><br>io = PetscBinaryIO.PetscBinaryIO()<br>io.writeBinaryFile('Vector_b.dat', [vec,])<br>io.writeBinaryFile('Matrix_A.dat',[A_matrix,])</div><div><br></div><div>This will output two PETSc ready files Vector_b.dat and Matrix_A.dat that you can exploit further with Vec/MatLoad or similar. <br></div><div><br></div><div>Cheers,</div><div>S<br></div><div><br></div><div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr">On Thu, Nov 8, 2018 at 7:53 PM Jed Brown <<a href="mailto:jed@jedbrown.org">jed@jedbrown.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Sal Am <<a href="mailto:tempohoper@gmail.com" target="_blank">tempohoper@gmail.com</a>> writes:<br>
<br>
> Yes I was just hoping there'd be more than that.<br>
><br>
> I have tried using one of them as basis:<br>
><br>
> import PetscBinaryIO<br>
> import numpy as np<br>
> import scipy.sparse<br>
><br>
> b_vector = np.array(np.fromfile('Vector_b.bin'<br>
> ,dtype=np.dtype((np.float64,2))));<br>
> A_matrix =<br>
> np.array(np.fromfile('Matrix_A_cmplx.bin',dtype=np.dtype((np.float64,2))));<br>
> A_indexs = np.array(np.fromfile('Matrix_A_int.bin'<br>
> ,dtype=np.dtype((np.int32 ,2))));<br>
><br>
> b_vector = b_vector[:,0] + 1j*b_vector[:,1]<br>
> A_matrix = A_matrix[:,0] + 1j*A_matrix[:,1]<br>
><br>
> A_matrix = scipy.sparse.csr_matrix((A_matrix, (A_indexs[:,0]-1,<br>
> A_indexs[:,1]-1)))<br>
><br>
> vec = b_vector.view(PetscBinaryIO.Vec)<br>
><br>
> io = PetscBinaryIO.PetscBinaryIO()<br>
> io.writeBinaryFile('Vector_b.dat', [vec,])<br>
> io.writeMatSciPy('Matrix_A.dat', A_matrix)<br>
<br>
The "fh" argument to writeMatSciPy is a file handle. You can use<br>
writeBinaryFile with matrices, in one file or separate files.<br>
<br>
> I was able to do it convert and read the Vector_b in fine.<br>
> So what I am having trouble with is actually converting the<br>
> scipy.sparse.csr_matrix into the PETSc format. I thought the writeMatSciPy<br>
> would do that by calling writeMatSparse?<br>
> Maybe I have misunderstood something here?<br>
><br>
> All the best,<br>
> S<br>
><br>
> On Thu, Nov 8, 2018 at 10:10 AM Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> wrote:<br>
><br>
>> On Thu, Nov 8, 2018 at 4:56 AM Sal Am via petsc-users <<br>
>> <a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>> wrote:<br>
>><br>
>>> Thanks, I missed that,<br>
>>><br>
>>> I do care about scalability, the long term plan is to make it fully<br>
>>> compatible on a cluster so I guess I would need to convert it to PETSc<br>
>>> binary format before reading it in.<br>
>>><br>
>>> Is this the file you were referring to<br>
>>> <a href="https://github.com/erdc/petsc-dev/blob/master/bin/pythonscripts/PetscBinaryIO.py" rel="noreferrer" target="_blank">https://github.com/erdc/petsc-dev/blob/master/bin/pythonscripts/PetscBinaryIO.py</a><br>
>>><br>
>>> Cannot seem to find docs on how to use it, do you happen to have any<br>
>>> examples?<br>
>>><br>
>><br>
>> The documentation is at the top of the file. You can see it by clicking<br>
>> the link.<br>
>><br>
>> Matt<br>
>><br>
>><br>
>>> All the best,<br>
>>> S<br>
>>><br>
>>><br>
>>><br>
>>><br>
>>> On Wed, Nov 7, 2018 at 4:07 PM Jed Brown <<a href="mailto:jed@jedbrown.org" target="_blank">jed@jedbrown.org</a>> wrote:<br>
>>><br>
>>>> Please always use "reply-all" so that your messages go to the list.<br>
>>>> This is standard mailing list etiquette. It is important to preserve<br>
>>>> threading for people who find this discussion later and so that we do<br>
>>>> not waste our time re-answering the same questions that have already<br>
>>>> been answered in private side-conversations. You'll likely get an<br>
>>>> answer faster that way too.<br>
>>>><br>
>>>> Sal Am <<a href="mailto:tempohoper@gmail.com" target="_blank">tempohoper@gmail.com</a>> writes:<br>
>>>><br>
>>>> > Thank you Jed for the quick response!<br>
>>>> ><br>
>>>> >><br>
>>>> >> Yes, of course the formats would have to match. I would recommend<br>
>>>> >> writing the files in an existing format such as PETSc's binary format.<br>
>>>> >><br>
>>>> ><br>
>>>> > Unfortunately I do not think I can change the source code to output<br>
>>>> those<br>
>>>> > two files in PETSc format and it would probably take a very long time<br>
>>>> > converting everything into PETSc (it is not even my code).<br>
>>>><br>
>>>> File-based workflows are very often bottlenecks. You can also use any<br>
>>>> convenient software (e.g., Python or Matlab/Octave) to convert your<br>
>>>> custom binary formats to PETSc binary format (see PetscBinaryIO provided<br>
>>>> with PETSc), at which point you'll be able to read in parallel. If you<br>
>>>> don't care about scalability or only need to read once, then you can<br>
>>>> write code of the type you propose.<br>
>>>><br>
>>>> > Do you have any other suggestions on how to read in those two complex<br>
>>>> > binary files? Also why would it be more difficult to parallelise as<br>
>>>> > I thought getting the two files in PETSc vector format would allow me<br>
>>>> to<br>
>>>> > use the rest of PETSc library the usual way?<br>
>>>> ><br>
>>>> > Kind regards,<br>
>>>> > S<br>
>>>> ><br>
>>>> ><br>
>>>> > On Mon, Nov 5, 2018 at 4:49 PM Jed Brown <<a href="mailto:jed@jedbrown.org" target="_blank">jed@jedbrown.org</a>> wrote:<br>
>>>> ><br>
>>>> >> Sal Am via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>> writes:<br>
>>>> >><br>
>>>> >> > Hi,<br>
>>>> >> ><br>
>>>> >> > I am trying to solve a Ax=b complex system. the vector b and<br>
>>>> "matrix" A<br>
>>>> >> are<br>
>>>> >> > both binary and NOT created by PETSc. So I keep getting error<br>
>>>> messages<br>
>>>> >> that<br>
>>>> >> > they are not correct format when I read the files with<br>
>>>> >> PetscViewBinaryOpen,<br>
>>>> >> > after some digging it seems that one cannot just read a binary file<br>
>>>> that<br>
>>>> >> > was created by another software.<br>
>>>> >><br>
>>>> >> Yes, of course the formats would have to match. I would recommend<br>
>>>> >> writing the files in an existing format such as PETSc's binary format.<br>
>>>> >> While the method you describe can be made to work, it will be more<br>
>>>> work<br>
>>>> >> to make it parallel.<br>
>>>> >><br>
>>>> >> > How would I go on to solve this problem?<br>
>>>> >> ><br>
>>>> >> > More info and trials:<br>
>>>> >> ><br>
>>>> >> > "matrix" A consists of two files, one that contains row column index<br>
>>>> >> > numbers and one that contains the non-zero values. So what I would<br>
>>>> have<br>
>>>> >> to<br>
>>>> >> > do is multiply the last term in a+b with PETSC_i to get a real +<br>
>>>> >> imaginary<br>
>>>> >> > vector A.<br>
>>>> >> ><br>
>>>> >> > vector b is in binary, so what I have done so far (not sure if it<br>
>>>> works)<br>
>>>> >> is:<br>
>>>> >> ><br>
>>>> >> > std::ifstream input("Vector_b.bin", std::ios::binary );<br>
>>>> >> > while (input.read(reinterpret_cast<char*>(&v), sizeof(float)))<br>
>>>> >> > ierr =<br>
>>>> VecSetValues(u,1,&iglobal,&v,INSERT_VALUES);CHKERRQ(ierr);<br>
>>>> >> ><br>
>>>> >> > where v is a PetscScalar.<br>
>>>> >> ><br>
>>>> >> > Once I am able to read both matrices I think I can figure out the<br>
>>>> solvers<br>
>>>> >> > to solve the system.<br>
>>>> >> ><br>
>>>> >> > All the best,<br>
>>>> >> > S<br>
>>>> >><br>
>>>><br>
>>><br>
>><br>
>> --<br>
>> What most experimenters take for granted before they begin their<br>
>> experiments is infinitely more interesting than any results to which their<br>
>> experiments lead.<br>
>> -- Norbert Wiener<br>
>><br>
>> <a href="https://www.cse.buffalo.edu/~knepley/" rel="noreferrer" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br>
>> <<a href="http://www.cse.buffalo.edu/~knepley/" rel="noreferrer" target="_blank">http://www.cse.buffalo.edu/~knepley/</a>><br>
>><br>
</blockquote></div>