if the file is stored in the PETSc format, you can use PETSc to pull the file in using MPIIO, which should be (hopefully) faster.<div><br></div><div><a href="http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Viewer/PetscViewerBinarySetMPIIO.html">http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Viewer/PetscViewerBinarySetMPIIO.html</a></div>
<div><br></div><div>Feel free to pop these questions to petsc-users (cc'd), there are more of them and they usually know more than me :)</div><div><br></div><div>-Aron<br><br><div class="gmail_quote">On Sat, Jan 8, 2011 at 2:21 PM, Zuhair Khayyat <span dir="ltr"><<a href="mailto:zuhair.khayyat@kaust.edu.sa">zuhair.khayyat@kaust.edu.sa</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div dir="ltr">Dear Aron,<br><br>Currently I am using PETSc for my research to implement a graph mining algorithm on a large cluster, and I would like ask you some questions due to your experience with this tool.<br>
<br>Have you ever tried to optimize allocating a very large matrix from input file in parallel? I have a very large file (around 10 GB) and it takes too long to allocate the matrix through the main node. I am planning to implement a distributed parallel file reader that split the original file and make each node reads it separately into a common matrix.<br>
<br>Is there and other tools that are comparable to PETSc in which they have parallel file reader?<br><br>Thank you for your help<br><br>Regards,<br>Zuhair Khayyat<br></div>
</blockquote></div><br></div>