<html><body><div style="color:#000; background-color:#fff; font-family:times new roman, new york, times, serif;font-size:12pt"><div><span>Thank you for the clarification Jon.<br></span></div><div><br></div> <div style="font-family: times new roman, new york, times, serif; font-size: 12pt;"> <div style="font-family: times new roman, new york, times, serif; font-size: 12pt;"> <div dir="ltr"> <font size="2" face="Arial"> <hr size="1"> <b><span style="font-weight:bold;">From:</span></b> Jonathan Monette <jonmon@mcs.anl.gov><br> <b><span style="font-weight: bold;">To:</span></b> Emalayan Vairavanathan <svemalayan@yahoo.com> <br><b><span style="font-weight: bold;">Cc:</span></b> "swift-devel@ci.uchicago.edu Devel" <swift-devel@ci.uchicago.edu>; MosaStore <mosastore@googlegroups.com>; Michael Wilde <wilde@mcs.anl.gov>; matei <matei@ece.ubc.ca> <br> <b><span style="font-weight: bold;">Sent:</span></b> Thursday, 15 March 2012
5:55 AM<br> <b><span style="font-weight: bold;">Subject:</span></b> Re: [Swift-devel] Mosaswift update<br> </font> </div> <br><div id="yiv1375773472"><div><div>I do not think it will matter since all the intermediate files will be mapped onto mosa but you can change the paths in those generated files to point to whatever you want. Then you can experiment with having the starting and final files in mosa if you want. <br><br>On Mar 15, 2012, at 3:30, Emalayan Vairavanathan <<a rel="nofollow" ymailto="mailto:svemalayan@yahoo.com" target="_blank" href="mailto:svemalayan@yahoo.com">svemalayan@yahoo.com</a>> wrote:<br><br></div><div></div><blockquote type="cite"><div><div style="color:#000;background-color:#fff;font-family:times new roman, new york, times, serif;font-size:12pt;"><div><span>Hi Jon and All,</span></div><div><br></div><div>MosaStore+Swift integration is done successfully. This is really a great
news.<br></div><div><br></div><div><span>I successfully ran the setup provided by Jon with MosaStore+Swift on Surveyor (on 1, 3 , 5 and 64 nodes with different number of jobs varying from 1 to 1024).</span></div><div><br><span></span></div><div><span>Jon and Mike thank you again for your time in helping me to get this far.</span></div><div><br><span></span></div><div><span style="font-weight:bold;">A quick question:</span> The setup creates the swift working directory in GPFS but stores the intermediate files in MosaStore. Where should we actually have the swift-working directory (in GPFS / Mosa)? and Does this matter at all in case if I am going to use Mosa as an intermediate storage?
<br><span></span></div><span><br>(As a next step I will be scripting our pipeline benchmark on swift and run at large scale to get some initial performance numbers. Also I will be parallely working to get Montage and ModFTDock running on BG/P with swift-CDM and MosaStore.)<br><br>Regards<br>Emalayan<br></span><div><br></div> <div style="font-family:times new roman, new york, times, serif;font-size:12pt;"> <div style="font-family:times new roman, new york, times, serif;font-size:12pt;"> <div dir="ltr"> <font size="2" face="Arial"> <hr size="1"> <b><span style="font-weight:bold;">From:</span></b> Jonathan Monette <<a rel="nofollow" ymailto="mailto:jonmon@mcs.anl.gov" target="_blank" href="mailto:jonmon@mcs.anl.gov">jonmon@mcs.anl.gov</a>><br> <b><span style="font-weight:bold;">To:</span></b> Emalayan Vairavanathan <<a rel="nofollow" ymailto="mailto:svemalayan@yahoo.com" target="_blank"
href="mailto:svemalayan@yahoo.com">svemalayan@yahoo.com</a>> <br><b><span style="font-weight:bold;">Cc:</span></b> "<a rel="nofollow" ymailto="mailto:mosastore@googlegroups.com" target="_blank" href="mailto:mosastore@googlegroups.com">mosastore@googlegroups.com</a>" <<a rel="nofollow" ymailto="mailto:mosastore@googlegroups.com" target="_blank" href="mailto:mosastore@googlegroups.com">mosastore@googlegroups.com</a>>; Justin M Wozniak <<a rel="nofollow" ymailto="mailto:wozniak@mcs.anl.gov" target="_blank" href="mailto:wozniak@mcs.anl.gov">wozniak@mcs.anl.gov</a>>; "<a rel="nofollow" ymailto="mailto:swift-devel@ci.uchicago.edu" target="_blank" href="mailto:swift-devel@ci.uchicago.edu">swift-devel@ci.uchicago.edu</a> Devel"
<<a rel="nofollow" ymailto="mailto:swift-devel@ci.uchicago.edu" target="_blank" href="mailto:swift-devel@ci.uchicago.edu">swift-devel@ci.uchicago.edu</a>> <br> <b><span style="font-weight:bold;">Sent:</span></b> Wednesday, 14 March 2012 2:29 PM<br> <b><span style="font-weight:bold;">Subject:</span></b> Re: [Swift-devel] Mosaswift update<br> </font> </div> <br><div id="yiv1375773472"><div>The working example is located in /home/jonmon/Workspace/Swift/mosaswift-test<div><br></div><div>There is a README there that provides a little bit of information.</div><div><br></div><div>Quick version to run it:</div><div>1) start your workers</div><div>2) ./setup.sh</div><div>3) ./run.sh</div><div><br></div><div>The directory final will have 5 files if it completed without error. The content of the files will look like:</div><div>Hello world</div><div>Hello from <number></div><div><br><div><div>On Mar 14, 2012, at 4:03 PM, Emalayan Vairavanathan
wrote:</div><br class="yiv1375773472Apple-interchange-newline"><blockquote type="cite"><div><div style="color:#000;background-color:#fff;font-family:times new roman, new york, times, serif;font-size:12pt;"><div><span>Hi Jon,</span></div><div><br><span></span></div><div><span>Great, Thank you very much. Could you please send me the setup ? <br></span></div><div><br><span></span></div><div><span>Thank you</span></div><div><span>Emalayan<br></span></div><div><br></div> <div style="font-family:times new roman, new york, times, serif;font-size:12pt;"> <div style="font-family:times new roman, new york, times, serif;font-size:12pt;"> <div dir="ltr"> <font size="2" face="Arial"> <hr size="1"> <b><span style="font-weight:bold;">From:</span></b> Jonathan Monette <<a rel="nofollow" ymailto="mailto:jonmon@mcs.anl.gov" target="_blank" href="mailto:jonmon@mcs.anl.gov">jonmon@mcs.anl.gov</a>><br> <b><span style="font-weight:bold;">To:</span></b> Justin M
Wozniak <<a rel="nofollow" ymailto="mailto:wozniak@mcs.anl.gov" target="_blank" href="mailto:wozniak@mcs.anl.gov">wozniak@mcs.anl.gov</a>> <br><b><span style="font-weight:bold;">Cc:</span></b> "<a rel="nofollow" ymailto="mailto:swift-devel@ci.uchicago.edu" target="_blank" href="mailto:swift-devel@ci.uchicago.edu">swift-devel@ci.uchicago.edu</a> Devel" <<a rel="nofollow" ymailto="mailto:swift-devel@ci.uchicago.edu" target="_blank" href="mailto:swift-devel@ci.uchicago.edu">swift-devel@ci.uchicago.edu</a>>; MosaStore
<<a rel="nofollow" ymailto="mailto:mosastore@googlegroups.com" target="_blank" href="mailto:mosastore@googlegroups.com">mosastore@googlegroups.com</a>> <br> <b><span style="font-weight:bold;">Sent:</span></b> Wednesday, 14 March 2012 2:00 PM<br> <b><span style="font-weight:bold;">Subject:</span></b> Re: [Swift-devel] Mosaswift update<br> </font> </div> <br>So that actually fixed the issue. So we have a working example of using CDM and intermediate file storage that Emalayan can use for testing his mosa store set up.<br><br>On Mar 14, 2012, at 3:05 PM, Jonathan Monette wrote:<br><br>> Oh, thanks, didn't catch that. Trying another run right now but I think someone is using all of surveyor right now, but manual coaster service job is stuck in the queue state.<br>> <br>> <br>> On Mar 14, 2012, at 3:02 PM, Justin M Wozniak wrote:<br>> <br>>> <br>>> Ok, then it's a typo on readlinE .<br>>> <br>>> On
Wed, 14 Mar 2012, Jonathan Monette wrote:<br>>> <br>>>> I am not sure. I thought readlink returned the
absolute path of the link which is what I wanted to use. I am not sure how readlink behaves on the compute nodes. This was my first attempt at getting around the link but I do not think this is working.<br>>>> <br>>>> On Mar 14, 2012, at 2:43 PM, Justin M Wozniak wrote:<br>>>> <br>>>>> On Wed, 14 Mar 2012, Jonathan Monette wrote:<br>>>>> <br>>>>>> I say semi working because I intended the script to build up output during the stages on the swift run and the final output would show that it was stored temporarily in /tmp. This is not happening as the final output for the file contains only the last line(a Hello from <id> line). I believe this is due to how CDM using symlinks for the files that is matched, so a symlink is being overwritten instead of at the file it points to. I ran into this problem in the past(but this is documented in the user guide) and just
need verify that this is indeed the issue.<br>>>>> <br>>>>> In the past, the application unlinked the link. It looks like you are appending to the link.<br>>>>> <br>>>>> infile=`readlink $1`<br>>>>> outfile=`readline $2`<br>>>>> cat $infile >> $outfile<br>>>>> <br>>>>> Is there a case in which the readlink affects the behavior of this sequence?<br>>>>> <br>>>>> Does readline work correctly on the BG/P compute node?<br>>>>> <br>>>>> Justin<br>>>>> <br>>>>> --<br>>>>> Justin M Wozniak<br>>>> <br>>>> <br>>> <br>>> -- <br>>> Justin M Wozniak<br>> <br>> _______________________________________________<br>> Swift-devel mailing list<br>> <a rel="nofollow" ymailto="mailto:Swift-devel@ci.uchicago.edu"
target="_blank" href="mailto:Swift-devel@ci.uchicago.edu">Swift-devel@ci.uchicago.edu</a><br>> <a rel="nofollow" target="_blank" href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel</a><br><br>-- <br>You received this message because you are subscribed to the Google Groups "MosaStore" group.<br>To post to this group, send email to <a rel="nofollow" ymailto="mailto:mosastore@googlegroups.com" target="_blank" href="mailto:mosastore@googlegroups.com">mosastore@googlegroups.com</a>.<br>To unsubscribe from this group, send email to mosastore+<a rel="nofollow" ymailto="mailto:unsubscribe@googlegroups.com" target="_blank" href="mailto:unsubscribe@googlegroups.com">unsubscribe@googlegroups.com</a>.<br>For more options, visit this group at <a rel="nofollow" target="_blank"
href="http://groups.google.com/group/mosastore?hl=en">http://groups.google.com/group/mosastore?hl=en</a>.<br><br><br><br> </div> </div> </div></div>_______________________________________________<br>Swift-devel mailing list<br><a rel="nofollow" ymailto="mailto:Swift-devel@ci.uchicago.edu" target="_blank" href="mailto:Swift-devel@ci.uchicago.edu">Swift-devel@ci.uchicago.edu</a><br><a rel="nofollow" target="_blank" href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel</a><br></blockquote></div><br></div></div>
-- <br>
You received this message because you are subscribed to the Google Groups "MosaStore" group.<br>
To post to this group, send email to <a rel="nofollow" ymailto="mailto:mosastore@googlegroups.com" target="_blank" href="mailto:mosastore@googlegroups.com">mosastore@googlegroups.com</a>.<br>
To unsubscribe from this group, send email to <a rel="nofollow" ymailto="mailto:mosastore+unsubscribe@googlegroups.com" target="_blank" href="mailto:mosastore+unsubscribe@googlegroups.com">mosastore+unsubscribe@googlegroups.com</a>.<br>
For more options, visit this group at <a rel="nofollow" target="_blank" href="http://groups.google.com/group/mosastore?hl=en">http://groups.google.com/group/mosastore?hl=en</a>.<br>
</div><br><br> </div> </div> </div></div></blockquote></div>
-- <br>
You received this message because you are subscribed to the Google Groups "MosaStore" group.<br>
To post to this group, send email to mosastore@googlegroups.com.<br>
To unsubscribe from this group, send email to mosastore+unsubscribe@googlegroups.com.<br>
For more options, visit this group at http://groups.google.com/group/mosastore?hl=en.<br>
</div><br><br> </div> </div> </div></body></html>