Hi all,<div><br></div><div>Let me join this conversation. I also "suffer" from these doubts. </div><div>In my case, I have an application in two versions, Windows (NTFS) and Linux (FAT32) and I have first implemented the first approach (make one separated copy for each machine).</div>
<div><br></div><div>But recently, I started to deal with bigger files (200Mb ~ 1Gb) and this became very inefficiently. Actually, the reason I suspect is that even we have multiple processes, the hard disk device that is responsible for manage all these readings is just one. In other words, this operation is intrinsically sequential and became a bottleneck (am I right?).</div>
<div><br></div><div>I didn't changed my implementation yet, but I was thinking to move to the second approach (rank 0 reads and BCast the info) expecting to have better results.</div><div><br></div><div>Does anyone have any experience?</div>
<div><br></div><div>Actually I am not sure if this will be better. I understand that MPI uses sockets to pass all messages and an natural question is if this operation is faster than reading from files?</div><div><br></div>
<div>Best regards,</div><div>Luiz</div><div><br></div><div><div class="gmail_quote">On Wed, Oct 22, 2008 at 12:10 AM, Rajeev Thakur <span dir="ltr"><<a href="mailto:thakur@mcs.anl.gov">thakur@mcs.anl.gov</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">How big is the file? What kind of file system is it on?<br>
<font color="#888888"><br>
Rajeev<br>
</font><div><div></div><div class="Wj3C7c"><br>
> -----Original Message-----<br>
> From: <a href="mailto:owner-mpich-discuss@mcs.anl.gov">owner-mpich-discuss@mcs.anl.gov</a><br>
> [mailto:<a href="mailto:owner-mpich-discuss@mcs.anl.gov">owner-mpich-discuss@mcs.anl.gov</a>] On Behalf Of<br>
> Kamaraju Kusumanchi<br>
> Sent: Tuesday, October 21, 2008 8:27 PM<br>
> To: <a href="mailto:mpich-discuss@mcs.anl.gov">mpich-discuss@mcs.anl.gov</a><br>
> Subject: [mpich-discuss] read the same file on all processes<br>
><br>
> Hi all,<br>
><br>
> I have a file which needs to be read on all the processes<br>
> of an MPI job. If I read the same file simultaneously on all<br>
> the processes, will it cause any problems?<br>
><br>
> I can think of two other options such as<br>
><br>
> - make multiple copies of the same file and read a separate<br>
> file on different processes<br>
> - read the file on rank 0 process, then use MPI_Bcast and<br>
> transfer the contents across the remaining processes.<br>
><br>
> Which approach should be preferred? I am thinking this<br>
> must be something encountered by others. So, if there is a<br>
> book/web page which explains these kind of things, a pointer<br>
> to them would be most appreciated.<br>
><br>
> regards<br>
> raju<br>
><br>
><br>
<br>
</div></div></blockquote></div><br></div>