<div dir="ltr">Dear Aron,<br><br>Actually the file is an output of another application, and is stored in a simple text file. I have read about MPIIO, however couldn't figure out if it is a parallel file reader or not.<br>
<br>Have you ever tried or worked on a parallel file reader to a common global matrix in PETSc, where each processor has part of the file? Thank you<br><br>Regards,<br>Zuhair Khayyat<br><br><div class="gmail_quote">On Sat, Jan 8, 2011 at 8:42 PM, Aron Ahmadia <span dir="ltr"><<a href="mailto:aron.ahmadia@kaust.edu.sa">aron.ahmadia@kaust.edu.sa</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">if the file is stored in the PETSc format, you can use PETSc to pull the file in using MPIIO, which should be (hopefully) faster.<div>
<br></div><div><a href="http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Viewer/PetscViewerBinarySetMPIIO.html" target="_blank">http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Viewer/PetscViewerBinarySetMPIIO.html</a></div>
<div><br></div><div>Feel free to pop these questions to petsc-users (cc'd), there are more of them and they usually know more than me :)</div><div><br></div><div><font color="#888888">-Aron</font><div><div></div><div class="h5">
<br><br><div class="gmail_quote">On Sat, Jan 8, 2011 at 2:21 PM, Zuhair Khayyat <span dir="ltr"><<a href="mailto:zuhair.khayyat@kaust.edu.sa" target="_blank">zuhair.khayyat@kaust.edu.sa</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div dir="ltr">Dear Aron,<br><br>Currently I am using PETSc for my research to implement a graph mining algorithm on a large cluster, and I would like ask you some questions due to your experience with this tool.<br>
<br>Have you ever tried to optimize allocating a very large matrix from input file in parallel? I have a very large file (around 10 GB) and it takes too long to allocate the matrix through the main node. I am planning to implement a distributed parallel file reader that split the original file and make each node reads it separately into a common matrix.<br>
<br>Is there and other tools that are comparable to PETSc in which they have parallel file reader?<br><br>Thank you for your help<br><br>Regards,<br>Zuhair Khayyat<br></div>
</blockquote></div><br></div></div></div>
</blockquote></div><br></div>