<div dir="ltr"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Hi Karthikeyan,</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">I'm guessing from the your mention of "processing from their original location" that Mike</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">must have been referring to staging mode direct.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">In this mode, swift points your applications to your data files which are on a shared filesystem</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">and avoid making copies. There are cases in which application access patterns are not</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">suitable for a shared filesystem, in such cases you can have swift make a copy from</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">the shared-filesystem to a local disk on the node. Since local disks are generally faster</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">than shared-filesystems, this might make more sense if your application is I/O intensive.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Here's a document that might help give you a clearer picture :</div><div class="gmail_default" style><font face="arial, helvetica, sans-serif"><a href="https://github.com/swift-lang/swift-k/blob/master/docs/staging/staging.asc">https://github.com/swift-lang/swift-k/blob/master/docs/staging/staging.asc</a></font><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Thanks,</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Yadu</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Feb 11, 2015 at 11:34 AM, Karthikeyan Balasubramanian <span dir="ltr"><<a href="mailto:karthikeyanb@uchicago.edu" target="_blank">karthikeyanb@uchicago.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">




<div>
<div style="direction:ltr;font-family:Tahoma;color:#000000;font-size:10pt">Hi,
<div><br>
</div>
<div>When jobs are run in Swift, it copies the data from the original location to the work folder for processing. Since our data are in the order of 15-30 GB, it adds unwanted overhead for the process. I recall from my early discussions with Michael Wilde that
 this could be optimized so that the files can be processed from their original location.</div>
<div><br>
</div>
<div>I am wondering as how should I proceed to achieve this optimization. Your inputs are greatly appreciated.</div>
<div><br>
</div>
<div>Thanks.</div>
<div>B.K.</div>
</div>
</div>

<br>_______________________________________________<br>
Swift-user mailing list<br>
<a href="mailto:Swift-user@ci.uchicago.edu">Swift-user@ci.uchicago.edu</a><br>
<a href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user" target="_blank">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user</a><br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr">Yadu Nand B<div><br></div></div></div>
</div>